As always, I take too long to reply to these threads. The multithreading problem is a hard one for this library. The reason being is that notification and access are not atomic unless you do Invokes rather than BeginInvokes. This means
that by the time the handler is able to read the data, the data may have changed again or if the clause of a query accesses the data multiple times then things could change between accesses. Imagine if you have a Where(obj => obj.Friend != null &&
obj.Friend.Name == "Foo"). If there is a context switch after the null check and someone sets the Friend property to null, then a null reference exception will be thrown. In my experience the best way to handle cross thread communication is
via message passing or ownership passing. Attempting to put in a locking mechanism for this would probably not work out either.
The best way to deal with this problem right now is to make your queries "live" on a single thread and marshal things to that thread. I could, write something that will be a cross thread collection change bridge so that you can define a query
on a background thread:
ReadOnlyObservableCollection<Person> ninjas = peopleCollection.Where( person => person.Title = "Ninja" );
Then, you would define a collection that you could then data bind to, or safely access from a different thread. (Let's just say this is the UI thread for argument's sake.)
ReadOnlyObservableCollection<Person> ninjasForThisThread = ninjas.MarshallToThisThread();
Under the covers, each thread would store it's own copy of the data. The query logic would only run on the background thread, and as things are added or removed from the collection on the background thread, they are then marshalled to the UI thread.
To the UI thread it would appear exactly the same.
Now... the only catch is that items in the collection that is on the background thread still cannot be modified on any other thread. For example, if you have two worker threads: thread A, and thread B, and you defined ninja query on thread A, you still
cannot update a person in the peopleCollection on Thread B without bad things happening. The reason being is laid out above.
What do you guys think? Coding the marshaller up shouldn't be too painful, but if the above solution won't help, let me know.
I am porting to Silverlight right now which is now going to be a new set of challenges as there is only one dispatcher in the whole system, and apps can't create Dispatchers for worker threads.
Another thing to consider for the property change marshalling. (does not affect collection change really)
Let's say you update 3000 objects that are being monitored by a query. If you put some kind of IsInvokeRequired in your OnPropertyChanged then you're in for 3000 dispatches across threads. So how do you batch these up? INotifyPropertyChanged
uses C# eventing system so I would need people to put some kind of "QueuePropertyChanged" into their property setters which I think sucks. On the other hand... ContinuousCollection has AddRange and RemoveRange which does do batching, but that
only helps for collection changes.
Before I go further... can anyone tell me what perf bottlenecks they've hit with CLINQ that has pushed these types of operations to a background thread? The reason I ask, is that the main app at my company that puts CLINQ through it's paces uses thousands
of objects with hundreds of live queries that all live on the UI thread. If you guys are exercising the system in a different way, perhaps I can change CLINQ to accommodate with
AK (your humble CLINQ narrator)