An Introduction to Elasticsearch (for .NET Developers) - 60 Mins

Slides from my talk "An Introduction to Elasticsearch (for .NET Developers)".

Elasticsearch is the world's leading search and analytics solution used by thousands of companies worldwide use cases, including search, analytics, monitoring and security information and event management. With an emphasis on speed, scale, and relevance, it's transforming how the world uses data. In this session, we'll learn about leveraging the power of Elasticsearch within .NET applications, utilising the Elasticsearch.NET client library. Join Steve to learn about the .NET client, how to install it in .NET applications and use it to begin indexing and searching documents. This session is aimed at software developers, looking to get started by combining the capabilities of Elasticsearch with their .NET applications. You'll leave with the core knowledge, required to begin using Elasticsearch using the .NET client.

Transcript

1. 1 Steve Gordon (Senior Engineer @ Elastic) @stevejgordon | stevejgordon.co.uk An Introduction to Elasticsearch (for .NET Developers) https://bit.ly/elasticsearch-net-intro
2. We build search solutions on a single stack Enterprise Search Observability Security
3. 4 Store, Search, & Analyze Visualize & Manage Ingest Elastic Stack SOLUTIONS Kibana Elasticsearch Beats Logstash SaaS Elastic cloud On-Prem Elastic cloud Enterprise Standalone Elastic cloud On Kubernetes Elastic Agent Serverless Elasticsearch Serverless
5. 6 Demo Elastic Cloud Deployment
6. 7 Basic Terminology CLUSTER A collection of one or more nodes (servers) that together hold your data and provide federated indexing and search capabilities across all nodes.
7. 8 Basic Terminology CLUSTER A single server that is part of your cluster, stores your data, and participates in the clusters indexing and search capabilities. NODE 1 NODE 2 NODE 3
8. 9 Basic Terminology CLUSTER A collection of documents that have somewhat similar characteristics. NODE 1 NODE 2 NODE 3 INDEX
9. 10 Basic Terminology CLUSTER Elasticsearch provides the ability to subdivide your index into multiple pieces called shards. NODE 1 NODE 2 NODE 3 INDEX SHARD (PRIMARY) P1 SHARD (PRIMARY) P2 SHARD (PRIMARY) P3 SHARD (REPLICA) R3 SHARD (REPLICA) R1 SHARD (REPLICA) R2
10. 11 Basic Terminology CLUSTER Elasticsearch provides the ability to subdivide your index into multiple pieces called shards. NODE 1 NODE 2 NODE 3 INDEX SHARD (PRIMARY) P1 SHARD (PRIMARY) P2 SHARD (PRIMARY) P3 SHARD (REPLICA) R3 SHARD (REPLICA) R1 SHARD (REPLICA) R2
11. 12 Basic Terminology CLUSTER The basic unit of information that can be indexed in JSON form. NODE 1 NODE 2 NODE 3 INDEX SHARD (PRIMARY) P1 SHARD (PRIMARY) P2 SHARD (PRIMARY) P3 SHARD (REPLICA) R3 SHARD (REPLICA) R1 SHARD (REPLICA) R2 DOC DOC DOC DOC DOC DOC DOC DOC DOC DOC DOC DOC
12. 13 JSON Document { "title": "This is a title for a blog post", "content": "This is the content for a really cool blog post about Elasticsearch", "category": "Elasticsearch", "date": "2023-04-14T09:27:00+0000", "isPublished": true, "upvotes": 512 }
13. 14 Field Mapping Dynamic mapping { "title": "This is a title for a blog post", "content": "This is the content for a really cool blog post about Elasticsearch", "category": "Elasticsearch", "date": "2023-04-14T09:27:00+0000", "isPublished": true, "upvotes": 512 } text field with a .keyword sub-field (ignore_above = 256)
14. 15 Field Mapping Dynamic mapping { "title": "This is a title for a blog post", "content": "This is the content for a really cool blog post about Elasticsearch", "category": "Elasticsearch", "date": "2023-04-14T09:27:00+0000", "isPublished": true, "upvotes": 512 } date field
15. 16 Field Mapping Dynamic mapping { "title": "This is a title for a blog post", "content": "This is the content for a really cool blog post about Elasticsearch", "category": "Elasticsearch", "date": "2023-04-14T09:27:00+0000", "isPublished": true, "upvotes": 512 } boolean field
16. 17 Field Mapping Dynamic mapping { "title": "This is a title for a blog post", "content": "This is the content for a really cool blog post about Elasticsearch", "category": "Elasticsearch", "date": "2023-04-14T09:27:00+0000", "isPublished": true, "upvotes": 512 } long field
17. 18 Explicit Mapping Field Types Aggregate metric Alias Arrays Binary Boolean Completion Date Date nanoseconds Dense vector Flattened Geopoint Geoshape Histogram IP Join Keyword Nested Numeric Object Percolator Point Range Rank feature Rank features Search-as-you-type Shape Text Token count Unsigned long Version
18. 19 19 HTTP Interface
19. 20 The Elasticsearch API in numbers • > 420 API endpoints (560 routes) • > 4000 data structures, including: ‒ 52 query types ‒ 79 aggregation types ‒ 35 field types
20. 21 Language Clients • .NET • Java • JavaScript • Ruby • Go • PHP • Perl • Python • Rust
21. 22 Previous Elasticsearch .NET Client v7 NEST High Level Client Strongly-typed requests, responses and Query DSL. Elasticsearch.Net Low Level Dependency free, unopinionated client. Handles transport and basic requests/responses.
22. 23 Problems with the v7 client • Hand written ‒ API is not always consistent ‒ A lot of maintenance work (400 endpoints and thousands of types!) • Legacy internalised JSON serialiser based on Utf8Json • 10 years of historical decisions
23. 24 Introducing Elastic.Clients.Elasticsearch • A new generation of the Elasticsearch client • Code Generated ‒ Based on a formal specification of the Elasticsearch API ‒ Most of the code is code-generated using Roslyn APIs • Uses System.Text.Json serializer • Built on a common Elastic.Transport layer • Removes some of the legacy of the past to create a cleaner API The new .NET client for v8.0
24. 25 Demo .NET Client
25. 26 Resources • https://bit.ly/elasticsearch-net-intro • github.com/elastic/elasticsearch-net • elastic.co/guide/en/elasticsearch/client/net-api/current • nuget.org/packages/NEST • nuget.org/packages/Elastic.Clients.Elasticsearch • github.com/stevejgordon/elasticsearch-examples • discuss.elastic.co
26. 27 Thank you! @stevejgordon