This thread looks to be a little on the old side and therefore may no longer be relevant. Please see if there is a newer thread on the subject and ensure you're using the most recent build of any software if your question regards a particular product.
This thread has been locked and is no longer accepting new posts, if you have a question regarding this topic please email us at support@mindscape.co.nz
|
Hi, I am writing an import batch that processes in blocks from a 100; private static void Import(string entityName, Func transformer) where T : Entity { var dof = new DatheonObjectsFetcher(DatheonCommunicator, null, Configuration.GetEntityIdFor(entityName)); Console.WriteLine("Start Import " + entityName + ":"); //using (var trans = new TransactionScope()) //{ Console.Write("\rTotal{0}, processing page {1}/{2}", dof.TotalFound, dof.CurrentPage, dof.Pages); while (dof.MoveNextPage(transformer)) { Console.Write("\rTotal{0}, processing page {1}/{2}", dof.TotalFound, dof.CurrentPage, dof.Pages); } UOW.SaveChanges(); //trans.Complete(); //} Console.WriteLine("End Import " + entityName + ":\n"); Console.ReadLine(); } Basically DatheonObjectsFetcher sets up the connection to the external source while MoveNextPage queries the next 100 results. The transformer is responsible of transforming the external source (an http xml response) to lightspeed entity's, the DatheonFetcherParser is a class that helps to easily parse each results xml element. public class DatheonFetcherParser { private Func ParseAttribute { get; set; } public Func ParseString { get; private set; } public Func ParseDateTime { get; private set; } public Func ParseBoolean { get; private set; } public DatheonFetcherParser(XElement element) { var l = element; this.ParseAttribute = new Func( (pre, n) => (l.Attribute(pre + "." + n) != null) ? l.Attribute(pre + "." + n).Value : null ); this.ParseString = new Func( (v) => this.ParseAttribute("Entity", v) ); this.ParseBoolean = new Func( (v) => this.ParseAttribute("Entity", v) == "TRUE" ); this.ParseDateTime = new Func( (s) => { string t = this.ParseString(s); DateTime dt; return (DateTime.TryParse(t, out dt)) ? dt : default(DateTime?); } ); } } I then call the import as followed var organisationType = FoundOrNewObject( Entity.Attribute("Name") == "International Broker", new OrganizationType() { Name = "International Broker" } ); Import("EEN_Broker", (parser) => { var organisationExp = Entity.Attribute("Name") == parser.ParseString("IRC_B_ORG_NAME"); var personExp = Entity.Attribute("FirstName") == parser.ParseString("IRC_B_Broker_FName") && Entity.Attribute("LastName") == parser.ParseString("IRC_B_Broker_LName"); var o = FoundOrNewObject(personExp, new Person() { FirstName = parser.ParseString("IRC_B_Broker_FName"), ..snap Organization = FoundOrNewObject(organisationExp, new Organization() { Name = parser.ParseString("IRC_B_ORG_NAME"), ..snap OrganizationType = organisationType }) }); if (o.EntityState == EntityState.Modified) UOW.SaveChanges(); return o; }); The FoundOrNewObject should return the object as stored in the database (if found or return the new object). This is however where it goes wrong. private static T FoundOrNewObject(QueryExpression expression, T newObject) where T : Entity { var q = new Query(typeof(T), expression); //var foundNo = myouw.Count(q); //if (foundNo > 1) //{ // var obj = myouw.Find(q).Take(1).First() ?? newObject; // return obj; // //} T o = UOW.Find(q).FirstOrDefault(); if (o != null) { return o; } o = newObject; UOW.Add(o); UOW.SaveChanges(); return o; } Eventhough Find finds doubles and returns the stored object it still inserts duplicates in the database. I am a bit at loss as to why. Infact the total rows in the database always equals that of what the import finds so no dupplicate is ever removed. Also right now I have to save changes after each object in order for Find to return doubles is there a better way to do this ? Ideally i want to query the database and the pending changes, and only SaveChanges() every block of 100 imported objects. FindOne fails in this case because it can return more then 1 result, I'm presuming the commented method is more performant then the current UOW.Find call, is this correct? |
|
|
The problem is that you are always new-ing up a Person and attaching it to the UOW. Remember that if you set an object's association, e.g. Person.Organisation or Organisation.OrganisationType, LightSpeed automatically joins that object to the associated object's unit of work. So your UOW is flowing from the OrganisationType, via the Organisation, to every Person. The fix is to either (a) create the Person only if you need to or (b) set the association only if you need to. Given the existing structure of your code, (a) is probably easier because you can just change FoundOrNewObject to take a Func<T> instead of a T, and invoke the Func<T> to create the entity only if you go down the "Find didn't return anything" path. Then you move that new Person { Organisation = ... } stuff inside a lambda. |
|