Elastic Search in C#

Elastic search:
Elastic search is one kind of full-text search and analytics engine for all type of data. It is a opensource and work based on Lucene Search Algorithm(i.e which is developed by using JAVA). Elastic search is a peer to peer based system so, nodes can communicate with one each other directly.

Why we use Elastic search:
1. To improve the performance of application while text based searching..
2. To get the almost matching results if user search typo mistakes..
3. To implement the Dashboards,logs etc..
4. While implementing application global search to fetch the records..etc…

Elastic search use cases:
1. If you search any article or query in stack-overflow,code-project,medium (i.e it’s contains Trillions of data..) etc.. you will get the results quickly even if any typo mistakes occurred..
2. We can use any domain like Healthcare,banking finance ,E-commerce etc..

Prerequisites for this session:
1. Elastic search server must installed or use cloud ES service API
2. SQL
3. visual studio with c#
4. elastic search chrome extension (i.e Optional).

Migration of existing SQL database to Elastic search Server:

We can utilized following ways..
1. while Inserting or after Inserting we can call Elastic search restful service to insert required fields in elastic search servers.
2. We can create the background job to monitor the SQL and pick the data from last inserted timestamp or auto increment ID..

I implemented this background job by using .Net core.whatever library’s required we need to download from Nuget .

Code:Migrate the Existing database to Elastic Search Server Using DSL query with C#:

Step: 1 Establish the Connection:
// 1. Connection URL’s elastic search
 var listOfUrls = new Uri[]
 {
 // here we can set multple connectionn URL’s…
 new Uri(“http://localhost:9200/”)
 };
StaticConnectionPool connPool = new StaticConnectionPool(listOfUrls);
 ConnectionSettings connSett = new ConnectionSettings(connPool);
 ElasticClient eClient = new ElasticClient(connSett);
Step: 2 Create a Index:
// 2. check the index exist or not 
 var checkResult = eClient.IndexExists(INDEX_NAME);
 if (!checkResult.Exists)
 {
 var createIndexResponse = eClient.CreateIndex(INDEX_NAME, c => c
 .Mappings(ms => ms.Map<Salesreport>(m => m.AutoMap()))
 );
 if (createIndexResponse.ApiCall.Success && createIndexResponse.IsValid)
 {
 // index is created successfully….
 }
 else
 {
 // fail log the exception further use
 var exception = createIndexResponse.OriginalException.ToString();
 var debugException = createIndexResponse.DebugInformation.ToString();
 }
 }
Step: 3 Get the last record of corresponding index with index type
var lastRecordResponse = eClient.Search<Salesreport>(s => s
 .Index(INDEX_NAME)
 .Type(INDEX_TYPE)
 .From(0)
 .Size(1).Sort(sr => sr.Descending(f => f.Salesid)));
Step: 4 Get All records from SQL
long salesRecordId = 0;
 var listofrecords = new List<Salesreport>();
 if (lastRecordResponse.Documents.Count >= 1)
 {
 var obj = lastRecordResponse.Documents;
 foreach (var item in obj)
 {
 salesRecordId = item.Salesid;
 }
listofrecords = GetAllRecords(salesRecordId);
}
 else
 {
 listofrecords = GetAllRecords(salesRecordId);
 }
Step: 5 Insert into Elastic server corresponding index
var bulkResponse = await Eclient.BulkAsync(b => b
 .Index(INDEX_NAME)
 .IndexMany(item));
if (bulkResponse.ApiCall.Success && bulkResponse.IsValid)
 {
 // success fully inserted…
 }
 else
 {
 // fail log the exception further use
 var exception = bulkResponse.OriginalException.ToString();
 var debugException = bulkResponse.DebugInformation.ToString();
 }

Small Use case for Elastic search VS SQL (Relational Database)

For example:
Let’s consider small use case like E-commerce Filed like amazon. if you search any product information from master table (i.e It’s contains trillions of data) and it contains the child tables. if you search Item-type=’snacks’ then we need to display the list of items related to snacks as well as Item-location,item price,Item type from different tables. In relational databases we use join the tables and retrieve the results from database. In relation databases (i.e trillions of data) we used to join parent, child tables and apply the where condition to retrieve the results. Some times it’s take more time to execute the relational databases query to fetch the results even if you apply the performance tuning related changes.. To overcome this type of scenario's we will use elastic search to improve the text based search results.

DB Design

I took almost 5 Million records in my database and the search the Item-type=’snacks’ using sql query and elastic search DSL query. I get the restults like below.
To Fetch 437560 records count using SQL — 1 min 35.98s
To Fetch 437560 records count using Elastic Search — 23.39s (<1 min)

For practical Session visit below link :

For full Source code visit below link:

RamanaReddyV/ElasticSearch

For Real-Time example: Stack-overflow

We will display the results based on analytics score of text based search word.If you know the ID Item we can fetch the item related details easily from Relational database’s.

Note: These all are low level basic things to use elastic search. If you need high level business need please follow the elastic search documentation.

==If you find any thing wrong please let me know i will update my self.==

Delete the Elastic search Indexes by using power shell