Skip to content

Commit 53abadb

Browse files
maumarmaumar
committed
full-text search docs
1 parent 2724195 commit 53abadb

File tree

1 file changed

+143
-0
lines changed
  • entity-framework/core/what-is-new/ef-core-10.0

1 file changed

+143
-0
lines changed

‎entity-framework/core/what-is-new/ef-core-10.0/whatsnew.md

+143
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,149 @@ EF10 requires the .NET 10 SDK to build and requires the .NET 10 runtime to run.
2121

2222
## Azure Cosmos DB for NoSQL
2323

24+
<a name="full-text-search-support"></a>
25+
26+
### Full-text search support
27+
28+
Azure Cosmos DB now offers support for [full-text search](/azure/cosmos-db/gen-ai/full-text-search). It enables efficient and effective text searches, as well as evaluating the relevance of documents to a given search query. It can be used in combination with vector search to improve the accuracy of responses in some AI scenarios.
29+
EF Core 10 is adding support for this feature allowing for modeling the database with full-text search enabled properties and using full-text search functions inside queries targeting Azure Cosmos DB.
30+
31+
<a name="full-text-search-model-configuration"></a>
32+
33+
#### Model configuration
34+
35+
A property can be configured inside `OnModelCreating` to use full-text search by enabling it for the property and defining a full-text index:
36+
37+
```c#
38+
public class Blog
39+
{
40+
...
41+
42+
public string Contents { get; set; }
43+
}
44+
45+
public class BloggingContext
46+
{
47+
...
48+
49+
protected override void OnModelCreating(ModelBuilder modelBuilder)
50+
{
51+
modelBuilder.Entity<Blog>(b =>
52+
{
53+
b.Property(x => x.Contents).EnableFullTextSearch();
54+
b.HasIndex(x => x.Contents).IsFullTextIndex();
55+
});
56+
}
57+
}
58+
```
59+
60+
> [!NOTE]
61+
> Configuring the index is not mandatory, but it is recommended as it greatly improves performance of full-text search queries.
62+
63+
Full-text search operations are language specific, using American English (`en-US`) by default. You can customize the language for individual properties as part of `EnableFullTextSearch` call:
64+
65+
```c#
66+
protected override void OnModelCreating(ModelBuilder modelBuilder)
67+
{
68+
modelBuilder.Entity<Blog>(b =>
69+
{
70+
b.Property(x => x.Contents).EnableFullTextSearch();
71+
b.HasIndex(x => x.Contents).IsFullTextIndex();
72+
b.Property(x => x.ContentsGerman).EnableFullTextSearch("de-DE");
73+
b.HasIndex(x => x.ContentsGerman).IsFullTextIndex();
74+
});
75+
}
76+
```
77+
78+
You can also set a default language for the container - unless overridden in the `EnableFullTextSearch` method, all full-text properties inside the container will use that language.
79+
80+
```c#
81+
protected override void OnModelCreating(ModelBuilder modelBuilder)
82+
{
83+
modelBuilder.Entity<Blog>(b =>
84+
{
85+
b.HasDefaultFullTextLanguage("de-DE");
86+
b.Property(x => x.ContentsEnglish).EnableFullTextSearch("en-US");
87+
b.HasIndex(x => x.ContentsEnglish).IsFullTextIndex();
88+
b.Property(x => x.ContentsGerman).EnableFullTextSearch();
89+
b.HasIndex(x => x.ContentsGerman).IsFullTextIndex();
90+
b.Property(x => x.TagsGerman).EnableFullTextSearch();
91+
b.HasIndex(x => x.TagsGerman).IsFullTextIndex();
92+
});
93+
}
94+
```
95+
96+
<a name="full-text-search-querying"></a>
97+
98+
#### Querying
99+
100+
As part of the full-text search feature, Azure Cosmos DB introduced several built-in functions which allow for efficient querying of content inside the full-text search enabled properties. These functions are: `FullTextContains`, `FullTextContainsAll`, `FullTextContainsAny`, which look for specific keyword or keywords and `FullTextScore`, which returns [BM25 score](https://en.wikipedia.org/wiki/Okapi_BM25) based on provided keywords.
101+
102+
> [!NOTE]
103+
> `FullTextScore` can only be used inside `OrderBy` to rank the documents based on the score.
104+
105+
EF Core exposes these functions as part of `EF.Functions` so they can be used in queries:
106+
107+
```c#
108+
var cosmosBlogs = await context.Blogs.Where(x => EF.Functions.FullTextContainsAll(x.Contents, "database", "cosmos")).ToListAsync();
109+
110+
var keywords = new string[] { "AI", "agent", "breakthrough" };
111+
var mostInteresting = await context.Blogs.OrderBy(x => EF.Functions.FullTextScore(x.Contents, keywords)).Take(5).ToListAsync();
112+
```
113+
114+
<a name="full-text-search-hybrid-search"></a>
115+
116+
#### Hybrid search
117+
118+
Full-text search can be used with vector search in the same query (i.e. hybrid search), by combining results of `FullTextScore` and `VectorDistance` functions. It can be done using the `RRF` function (Reciprocal Rank Fusion), which EF Core also provides inside `EF.Functions`:
119+
120+
```c#
121+
public class Blog
122+
{
123+
...
124+
125+
public float[] Vector { get; set; }
126+
public string Contents { get; set; }
127+
}
128+
129+
public class BloggingContext
130+
{
131+
...
132+
133+
protected override void OnModelCreating(ModelBuilder modelBuilder)
134+
{
135+
modelBuilder.Entity<Blog>(b =>
136+
{
137+
b.Property(x => x.Contents).EnableFullTextSearch();
138+
b.HasIndex(x => x.Contents).IsFullTextIndex();
139+
140+
b.Property(x => x.Vector).IsVectorProperty(DistanceFunction.Cosine, dimensions: 1536);
141+
b.HasIndex(x => x.Vector).IsVectorIndex(VectorIndexType.Flat);
142+
});
143+
}
144+
}
145+
146+
float[] myVector = ...
147+
var hybrid = await context.Blogs.OrderBy(x => EF.Functions.Rrf(
148+
EF.Functions.FullTextScore(x.Contents, "database"),
149+
EF.Functions.VectorDistance(x.Vector, myVector)))
150+
.Take(10)
151+
.ToListAsync();
152+
```
153+
154+
> [!TIP]
155+
> You can combine more than two scoring functions inside `Rrf` call, as well as using only `FullTextScore`, or only `VectorDistance`.
156+
157+
<a name="full-text-search-limitations"></a>
158+
159+
#### Limitations
160+
161+
Here are some current limitations of the full-text search using EF Core:
162+
163+
- `FullTextScore` and `RRF` functions can only be used inside `OrderBy` and are mutually exclusive with other forms of ordering.
164+
- EF Core can't create a container with full-text search or vector properties defined inside collection navigation - you can use those properties in queries, but the database has to be generated by other means (e.g. Microsoft.Azure.Cosmos SDK or using UI on Azure Portal).
165+
- Keyword arguments for `FullTextContainsAll`, `FullTextContainsAny` and `FullTextScore` have to be either constants or parameters. `FullTextContains` allows more complex expressions.
166+
24167
<a name="improved-model-evolution"></a>
25168

26169
### Improved experience when evolving the model

0 commit comments

Comments
 (0)