Archive for February, 2009

LightSpeed: Writing a custom Search Engine

LightSpeed includes the ability to use a custom full text search engine to more quickly and easily query your data in a soft manner. This has proved very helpful and saved many hours of work for website developers when they need to create a site search function.

By default, LightSpeed ships with a Lucene search engine implementation so that you can work with any of the many database engines LightSpeed supports and still get a very high quality, scalable and easy to use search capability.

This post is about helping developers who want to write their own custom search provider. Perhaps you don’t want to use Lucene or, perhaps you just have an existing search product you would like to leverage. LightSpeed makes it easy for you to plug in your own provider.

Configure a search engine

Configuration of the search engine capabilities is done at the LightSpeedContext level. For example, to configure a context to use the built in Lucene Search Engine, you could do the following in code to configure your LightSpeedContext (you can do this also in the application config, please check the help documentation on how to achieve this).

  LightSpeedContext.Default.SearchEngine = new SearchEngineBroker(new LuceneSearchEngine());
  LightSpeedContext.Default.SearchEngineFileLocation = "c:\searchindex";

Notice that there are two configuration properties for the search engine – the class that will act as the search engine, and the location of the index (if your search engine needs one). When you create your own provider, just plug the class in as the searchEngineClass and you’ll be ready to go!

The ISearchEngine interface

To create a search provider you must create a class that adheres to the ISearchEngine interface. Once a class conforms to this interface it can be assigned in the configuration as the Search Engine you wish to use.

  public interface ISearchEngine
  {
    IList<SearchResult> Search(string query, params string[] scopes);
    void Add(IndexKey indexKey, string data);
    void Update(IndexKey indexKey, string data);
    void Remove(IndexKey indexKey);
    void Optimize();
    void Clear();
    LightSpeedContext Context { get; set; }
    void BeginBulkAdd();
    void EndBulkAdd();
  }

But what should these methods do?

Most of these methods are self explanatory and there is significant intellisense support as well as documentation in the help file about this interface but here’s a break down of what each should do:

IList Search(string query, params string[] scopes);
Perform the actual search and return a collection of SearchResult items with details about the entities discovered. The scopes params collection is a string representation of the types that the user wishes to search over (and passed in as strings because some search repositories, Lucene for example, store everything as a string).

void Add(IndexKey indexKey, string data);
Add a single item to the index. The indexKey is a type that stores information about the scope, entity identity, etc. The data is the content that will be searched over and is a concatenation of the fields marked up as [Indexed] in the entity being added, or the overridden GetSearchData() method on the entity being indexed.

void Update(IndexKey indexKey, string data);
Update a single item in the index.

void Remove(IndexKey indexKey);
Remove a single item in the index.

void Optimize();
Optimize the index. This is useful for search engines that have an optimisation process that can run at the end of a indexing operation. Again, this was borrowed from the Lucene way of operating but if your custom search engine doesn’t have this operation then just don’t do anything in the method.

void Clear();
Reset the search store to an empty state.

LightSpeedContext Context { get; set; }
Manage the context that that this search engine is associated with. This provides access to the configuration of the context (such as the search engine index directory if you use one). LightSpeed will automatically assign the context to the property, so simply storing the value in a local variable is all you need to do.

void BeginBulkAdd();
Some search engines will have a mechanism for opening a search index writer so that it gets held open for bulk add operations. This is called when undertaking options like Rebuild() on the SearchEngineBroker for doing a mass re-creation of the search index.

void EndBulkAdd();
When undertaking a bulk add operation, there is often a need to close an index writing operation.

What’s next?

Have a go at writing a provider for youself! The best way to learn is to code something up yourself. If you’re not interested in writing your own the perhaps experiment with the built in Lucene search engine.

Click here to download a zip of the LuceneSearchEngine.cs (zipped) implementation as a reference for writing you own (but do note, it’s very specific to Lucene, your one will likely look considerably different internally). Let us know if you write one!

Add your comments
Author icon

Posted by John-Daniel Trask on 25 February 2009

Inheritance and database synchronisation

Tagged as LightSpeed

One of the big time-saving features in the LightSpeed designer is the ability to synchronise your database and your LightSpeed model — either by updating your model to reflect changes to your database or vice versa. It’s such a big time-saver that when people run into its limitations they feel quite disappointed. Until now there have been two major limitations on schema round-tripping: it doesn’t handle value objects, and it doesn’t handle inheritance. The reason for these limitations is that they break the easy mapping of entities to tables and properties to columns: value objects add extra columns to a table, and inheritance not only adds columns but changes the naive entity-to-table mapping. Of these two limitations, inheritance has been the one that most people run into.

We’re therefore pleased to announce that we’ve now added experimental support for inheritance to the synchronisation engine.

Synchronisation is now supported both for single-table inheritance scenarios, where all columns from across the hierarchy are collected into a single database table:

and for “concrete” table inheritance, where each non-abstract entity has its own database table which includes columns for both local and inherited properties:

(For more information about the two kinds of inheritance, see Inheritance in LightSpeed.)

Synchronisation still has a couple of minor limitations. Some of these are because of the fact that the database just doesn’t know about inheritance. For example, if you’re using single-table inheritance, and you’ve added a column in the database which is meaningful only on a derived type, we can’t know about that, so we’ll offer to add it to the base class. We hope at some point to add a “move to derived entity” command which will address this and will also with the initial creation of STI hierarchies from a database. Others are just due to resource limitations. For example, if you’re using concrete table inheritance, we still don’t handle concrete base classes, because of the complexity of reconciling database changes up and down the hierarchy — but the designer now supports marking base classes as abstract, which we expect to be the more common case anyway.

Been waiting for this since 2.0? Grab the latest nightly and take it for a whirl!

Add your comments
Author icon

Posted by Ivan Towlson on 24 February 2009

WPF Flow Diagrams 1.0 released

Tagged as Products, WPF Diagrams

We’re pleased to announce the availability of WPF Flow Diagrams, a 100% WPF library for creating and viewing flow diagrams.

WPF Flow Diagrams Screenshot

WPF Flow Diagrams provides viewing and editing of flow diagrams out of the box, with a couple of built-in styles, a baseline node library, drag-and-drop creation of nodes and connections, and a multitude of options for things like snapline display style, read-only status, zooming behaviour, and so on. The diagram library also has built-in support for printing, bitmap export and saving to XML.

Just as importantly, though, WPF Flow Diagrams also allows a great deal of customisation of the appearance of the diagrams. Everything is represented using templates, so you can bring all of WPF’s features to bear to create fully interactive, visually compelling user experiences. The class library exposes both standard implementations that you can reuse in your own designs, and extensibility hooks for when you want to go beyond what we provide. Full samples are included for creating custom styles and for extending the node library, as well as quick-start and feature explorer demos/tutorials.

Mindscape named #2 NZ Start-up

Tagged as News

I’m a bit late at posting this on our blog here but we were very pleased to see that Mindscape was named the #2 start-up for 2008 in the New Zealand Top 10 Start-up awards. It’s always nice to get recognised for the work we’re doing and are thankful to those that voted for us as well as our supporters and customers who have helped us to where we are :-)

Published on the New Zealand Herald Website:

Mindscape was founded in February 2007 by John-Daniel Trask, Jeremy Boyd and Andrew Peters with the intention of creating software products that “didn’t suck”. Tired of software feeling slow and bloated, they set about creating the tools that they, as software developers, would love to use – tools that were small, blazingly fast and effective.

Recognising the skills of the founders, Microsoft got in touch with Mindscape and enlisted their help to develop BackgroundMotion a technical best-practice open source application to aid other .NET developers worldwide in understanding how to best architect modern Web 2.0 websites on the .NET platform. This initial development work helped aid early profitability on top of the small seed capital the founders had provided at launch. Since this time Mindscape has released three products for software developers, continued to grow profits and revenue and has customers all over the world – including Fortune 50 companies.

Mindscape has achieved all of this with only three full-time employees and no external funding. They’ve been profitable since inception and continue to develop both the product and consulting sides of their business.

One thing I would like to add to this write up is to highlight the great work that Ivan has done as part of our team – both in leading our product development and helping out our customers in the support forums. His work certainly aided in achieving this award.

Thanks again for the votes.

Add your comments
Author icon

Posted by John-Daniel Trask on 19 February 2009

Running LightSpeed under medium trust

Tagged as LightSpeed

We’ve had some feedback recently about LightSpeed not running in medium trust, specifically hosted ASP.NET environments, so I’m pleased to announce some recent improvements in this area.

The main problem with running in medium trust has always been the way we get data into and out of entities. The technique we use is extremely fast, but requires permissions that aren’t allowed in medium trust.

So in future nightly builds we’re providing an option to fall back to a method that requires fewer privileges. You can choose this by setting the new LightSpeedContext.UseMediumTrustCompatibility flag:

LightSpeedContext.UseMediumTrustCompatibility = true;

(Notice that this is a static setting, not on a per-context basis.) The downside of setting this flag is that the “safe” method is a bit slower than the default method, so if your target deployment platform is medium trust then make sure you do your performance testing in medium trust.

Medium trust also imposes a couple of stylistic restrictions on the way you define your entities:

1. All fields must be declared read-write. We usually recommend that special fields such as DeletedOn or LockVersion, and value object fields, be declared readonly to prevent user code changing them, but in a medium trust environment that prevents LightSpeed from initialising them as it needs to. (By default the designer follows the recommended practice. If you want to use the designer, and your entities use Optimistic Concurrency Checking, Soft Delete, Track Create Time or Track Update Time, or your model contains value objects, you must set Medium Trust Compatibility at the model level.)

2. When implementing property setters, you must call the Set overload which takes the property name: that is, call Set(TField, TField, string) instead of Set(TField, TField). (The designer does this automatically for you.)

Finally, at present there are a couple of limitations when running in medium trust:

1. We’ve seen some problems during testing with configuration. You may need to set up your LightSpeedContext in code rather than config.

2. You can only use the core LightSpeed API: we don’t yet have LINQ running in medium trust.

We’ll keep you posted as we make further improvements in these areas. In the meantime, if you’re interested in medium trust, grab the latest nightly and give it a go.

Add your comments
Author icon

Posted by Ivan Towlson on 15 February 2009

Subscribe to our blog