This thread looks to be a little on the old side and therefore may no longer be relevant. Please see if there is a newer thread on the subject and ensure you're using the most recent build of any software if your question regards a particular product.
This thread has been locked and is no longer accepting new posts, if you have a question regarding this topic please email us at support@mindscape.co.nz
|
Due to a failure in one of my services, another service started queueing up database requests (its by design to handle short down-time issues). I noticed this too late and the queue had reached reached 461k records. Once the service that failed was fixed and restarted, the service that was queueing the records dumped all the queued records in one big SaveChanges. I personally didnt expect this to fail, however it did fail with this exception: Another issue is that it also changed the entity's state from New -> "something not New". This is because I check the EntityState to see if the entity has already been saved in the database. If it stayed at EntityState "New" then it should have kept failing at the SaveChanges, however the application continued as if the entities where saved to the database. The EntityState issue wouldnt have saved my application, but it made me wonder if the EntityState also changes if there is a database error. I will have to investigate that. Now I am not entirely sure why it failed, however if I have to guess then it looks like LightSpeed was building one massive SQL statement to insert all 461k records. The entities that LightSpeed was inserting where all using KeyTable as insert method and the database server was SQL Server 2008. And the LightSpeed was a nightly build from 15-03-2010. Anyhow, I am pretty sure that neither errors should happen. If needed then I can probably setup a test project for this. Regards, Jerremy |
|
|
|
|
The EntityState issue is fixed in recent nightly builds (22 April 2010 and above) -- if an error occurs during the save process (including database errors) then the EntityState will now be preserved so that the save can be retried. Your diagnosis of the "out of memory" error is roughly correct, though it's not quite the case that "LightSpeed was building one massive SQL statement" -- we actually build 461K separate SQL statements, but the upshot is still that we have to hold all the SQL in memory at once! I will take a look at whether we can modify this behaviour to enable us to cope with very large save batches. In the meantime a possible workaround would be for the service that pulls things off the queue to limit the number of things it pulls off in one go -- this might not be a bad idea anyway! Thanks for reporting these issues and for the detailed information! |
|
|
|
|
I'll upgrade to the latest nightly build to solve the entity state issue then. As for the out of memory part. I'll build in batch-insert for now (1024 or so records per savechanges). Its not common for the queue to build up as high as it did, so the batch-insert is easy enough to implement. I just didnt expect the insert to fail (only to take a shitload of time to complete ;) Talking about the nightly builds, is there a changes list somewhere? Its not common practice for me to keep up to date with the nightly builds, however an issue like the entity state issue would be a reason to upgrade. |
|
|
|
|
We don't currently publish a change list for the nightly builds. We're looking at ways in which we can make this happen without it becoming an admin burden for us. |
|
|
|
