✨ Enhancement

[Experimental] Render 🫎​ template route

by @Mubelotix in https://github.com/meilisearch/meilisearch/pull/5765

Introduces a new POST /render-template route that can be used to render any template or fragment on any input and associated renderRoute experimental feature that gates access to the route.

This route can be used to test document templates and fragments before and after having configured an embedder.

A body payload for the route is of the form:

{
  "template": /* templateTarget object */,
  "input": /* inputTarget object or null */
}

where template describes the template or fragment to render, and input describes what to use to render the template.

Upon calling this route, Meilisearch responds with:

{
  "template": "{{doc.text}}",
  "rendered": "template text after rendering using the input"
}

where template contains the unrendered base text of the document template, or the unrendered base JSON object of a fragment, and rendered contains the result of rendering the template of the chosen input.

If input is null in the request, then rendered is null in the response, and the route can be used solely to retrieve a template or fragment from the settings of an index.

Before calling the route

The API of this route is subject to change, so before calling this route, please enable the renderRoute experimental feature:

PATCH /experimental-features --json '{"renderRoute": true}'

Examples

1. Rendering a document from an index on a document template from an embedder of that index

// POST /render-template

{
  "template": {
    "kind": "documentTemplate",
    "indexUid": "movies",
    "embedder": "myMoviesEmbedder"
  },
  "input": {
    "kind": "indexDocument",
    "indexUid": "movies",
    "id": "2"
}

{
  "template": "A movie titled {{doc.title}} whose description starts with {{doc.overview|truncatewords:10}}",
  "rendered": "A movie titled Ariel whose description starts with Taisto Kasurinen is a Finnish coal miner whose father has..."
}

1. Rendering an inline document on a fragment from an embedder of an index

// POST /render-template

{
  "template": { 
    "kind": "indexingFragment", 
    "indexUid": "dogs", 
    "embedder": "multi",
    "fragment": "captionedImage" 
  },
  "input": { 
    "kind": "inlineDocument", 
    "inline": { // pass your document inline as a JSON object
      "kind": "dog",
      "name": "iko",
      "breed": "jack russell",
      "mime": "image/png",
      "image": "/9j/4AAQSk..."
    } 
  }
}

{
  "template": {
    "content": [
      {
        "type": "text",
        "text": "A picture of a {{doc.kind}} of breed {{doc.breed}}"
      },
      {
        "type": "image_base64",
        "image_base64": "data:{{doc.mime}};base64,{{doc.image}}"
      }
    ]
  },
  "rendered": {
    "content": [
      {
        "type": "text",
        "text": "A picture of a dog of breed jack russell"
      },
      {
        "type": "image_base64",
        "image_base64": "data:image/png;base64,/9j/4AAQSk..."
      }
    ]
  }
}

1. Rendering a search query on a search fragment from a multimodal embedder of an index

// POST /render-template
{
  "template": { 
    "kind": "searchFragment", 
    "indexUid": "testIndex", 
    "embedder": "testEmbedder",
    "fragment": "justBreed"
  },
  "input": {
    "kind": "inlineSearch",
    "inline": { // pass the search query inline
      "q": "unused",
      "media": {
        "name": "iko",
        "breed": "jack russell"
      },
      "filter": "ignored"
    }
  }
}

    {
      "template": "It's a {{ media.breed }}",
      "rendered": "It's a jack russell"
    }

1. Rendering an inline document on the document template from the chat settings of an index

// POST /render-template

{
  "template": {
    "kind": "chatDocumentTemplate",
    "indexUid": "movies"
	// no embedder to specify since chat document template is global to index
  },
  "input": {
    "kind": "indexDocument",
    "indexUid": "movies",
    "id": "2"
}

{
  "template": "{% for field in fields %}{% if field.is_searchable and field.value != nil %}{{ field.name }}: {{ field.value }}\n{% endif %}{% endfor %}",
  "rendered": "id: 2\ntitle: Ariel\noverview: Taisto Kasurinen is a Finnish coal miner whose father has just committed suicide and who is framed for a crime he did not commit. In jail, he starts to dream about leaving the country and starting a new life. He escapes from prison but things don't go as planned...\ngenres: DramaCrimeComedy\nposter: https://image.tmdb.org/t/p/w500/ojDg0PGvs6R9xYFodRct2kdI6wC.jpg\nrelease_date: 593395200\n"
}

1. Rendering a document from an index on an inline document template

// POST /render-template

{
  "template": {
    "kind": "inlineDocumentTemplate",
    "inline": "You can pass templates inline as well: nice to test them! {{doc.id}}"
  },
  "input": {
    "kind": "indexDocument",
    "indexUid": "movies",
    "id": "2"
}

{
  "template": "You can pass templates inline as well: nice to test them! {{doc.id}}",
  "rendered": "You can pass templates inline as well: nice to test them! 2"
}

1. Rendering an inline document on an inline indexing fragment

// POST /render-template

{
  "template": {
    "kind": "inlineFragment",
    "inline": {
      "json_maps": "supported for fragments",
      "any_string": "is in liquid format: {{doc.test}}"
    }
  },
  "input": {
     "kind": "inlineDocument",
    "inline": {
      "test": true
    }
  }
}

{
  "template": {
    "json_maps": "supported for fragments",
    "any_string": "is in liquid format: {{doc.test}}"
  },
  "rendered": {
    "json_maps": "supported for fragments",
    "any_string": "is in liquid format: true"
  }
}

[Experimental] Only support foreign filters on retrieval routes

by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/6446

Foreign filters are meant to be used in a retrieval context (search, get document...), but all the actions related to writing or modifying a document could have several unexpected behaviors if foreign filters are accepted.
We prefer forbidding the usage of this feature on the writing routes.

The following routes do not support Foreign-filter anymore:

• Edit documents by function: POST /indexes/{index_uid}/documents/edit
• Delete documents by filter: POST /indexes/{index_uid}/documents/delete
• Export to a remote Meilisearch: POST /export

Additional change: we now ensure that the experimental features are checked when parsing a filter

🪲 Bug fixes

• Support prefix search on words registered in the disableOnAttributes and disableOnNumbers settings by @antcybersec in https://github.com/meilisearch/meilisearch/pull/6432
• Add missing logs in search performance details @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/6457
• Ensure the index map budget is a multiplier of the OS page size by @genisis0x in https://github.com/meilisearch/meilisearch/pull/6454

🔒 Security

• Reduce risk of vulnerability exploits on GHA by @curquiza in https://github.com/meilisearch/meilisearch/pull/6451

🔩 Miscellaneous

• Replace the queueDocumentsFetch experimental feature with disableDocumentsFetchQueue converting the feature from an opt-in to an opt-out
  By @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/6456
• Bump and removes unused dependencies by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6444
• Fix Ollama embeddings changes to fix CI tests by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6450

❤️ Thanks again to @genisis0x and @antcybersec