Skip to main content
AI Chatbots

LLMQnA service configuration parameters

Metadata​

Defines metadata settings, including filtering conditions.

Metadata can be used in the following cases:

  1. When adding a file with specific metadata to the knowledge base, so that the file's content can be filtered (see use cases 2 and 3),
  2. When searching text only within fragments/files that contain specified metadata,
  3. When interacting with the assistant, to use only those fragments/files that contain the specified metadata.
"Metadata":{"shop":"true","services":"true","and_or":"and","exclude":{"ChunkNr":"8","public":"true"}}
  • For file searches, metatag values can be lists.
  • Filters by default are joined by logical operator 'or'. To use 'and' you have to specify it with the metatag 'and_or' (this tag is optional).
  • In markdown context files, #, ##, and ### are interpreted as text segment separators, with # xxx, ## yyy, and ### zzz being added to metadata as H1:xxx, H2:yyy, and H3:zzz. A line containing #### somekey:somevalue is added to metadata as somekey:somevalue, and a line with only #### signals a mandatory segment break.

Threshold​

Cosine similarity is used to determine the similarity threshold between the user's input text and segments in the knowledge base, excluding segments that exceed the specified value. Cosine similarity ranges from 0.0 to 2.0. The smaller the value, the more similar the segments are.

"Similarity":{"metric":"cosine_sim","threshold":"0.75"}

Debug​

Enables or disables debug mode. (If value is 'true' the complete prompt with context and history passed to LLM in "LLMinput” key, and the conversation history in "history" key are returned for chat method)

"Debug":"true"

LLM​

Specifies an alternative language model to be used if the web service is started with two available models — 'gpt-35-turbo' and 'gpt-4o'. (If an alternative model is not specified when starting the web service, this option has no effect.)

"LLM":"gpt-4o"

Different LLM​

Defines another LLM model.

"DifferentLLM":{"model":"gpt-4o","version":"xxx", "azure_deployment":"xxx", "azure_endpoint":"https://xxx.openai.azure.com/","api_key":"xxx"}

UseKeywords​

Values: 'true' or 'false' (default is 'true'). Enables keyword-based search and filtering. Keyword search is used in addition to semantic search when selecting context segments. Segments are filtered using keywords.

"UseKeywords":"true"

Language​

Language for keywords. Specifies the language for keyword processing.

"Language":"en"

HyDe (Experimental)​

Use Hypothetical Document. Enables the hypothetical document approach for improved query understanding. The value can be 'true' or 'false' (default is 'false'). When set to 'true', information is first generated using the LLM that could help answer the question. Then, semantically similar segments to this generated hypothetical segment are chosen from the vector database, instead of segments similar to the question.

"HyDe":"true"

ChunkSize​

Chunk size when splitting files. Defines how large file segments should be when splitting files for processing.

"ChunkSize":"1024"

SmartChunking​

Based on various text features (such as enumeration units, paragraphs, and other elements), the beginning of the next segment is determined. By default, the option is enabled with the value "true".

"SmartChunking":"false"

FilterReferences​

Filter References. Enables filtering of references in processed content. When set to 'true' (default is 'false'), only references that are semantically similar to the answer, based on cosine similarity, are returned. References that are not used in the response generation are excluded.

"FilterReferences":"true"

MultiQuestions (Experimental)​

If unable to answer, split the query into multiple questions and answer separately. Ensures that if a question is too complex, it is broken into smaller sub-questions for better results. When set to 'true' (default is 'false'), if the context does not provide an answer, the LLM will split the input into several parts and answer each part separately.

"MultiQuestions":"true"

ExtractPictures​

Extract images from PDFs into a separate folder. Ensures that images found in PDFs are saved separately. The getpictures method later automatically saves all images in the ‘Resources’ subview within the training view and creates links in the knowledge base document as references to the virtual assistant's resources folder.

"ExtractPictures":"true"

UseSummary (Experimental)​

Generate a file summary and add it to segment metadata, as well as before file segments when preparing the response context. The summary is created from the file and attached to the response context.

"UseSummary":"true"

Pdf2md​

to use different tool for extracting markdown from the pdf files; for getfilecontent and addfile methods

Examples of config parameter (if 'ExtractPictures' are 'true' the pictures from the pdf are saved as separate files and could be retrieved with the method getPictures):

{"ExtractPictures":"true","Pdf2md":{"type":"gemini","model":"gemini-2.0-flash","key":"<GOOGLE_API_KEY>"}}

{"Pdf2md":{"type":"llamacloudparse","key":"<LLAMA_CLOUD_API_KEY>"}}

{"ExtractPictures":"true","Pdf2md":{"type":"mistral","model":"mistral-ocr-latest","key":"<MISTRAL_API_KEY>"}}

{"ExtractPictures":"true","Pdf2md":{"type":"docintelligence","endpoint":"<AZURE_DOCINTELLIGENCE_ENDPOINT>","key":"<AZURE_DOCINTELLIGENCE_KEY>"}}

{"ExtractPictures":"true","Pdf2md":{"type":"markerpdf"}}

{"ExtractPictures":"true","Pdf2md":{"type":"pymupdf4llm"}}

{"ExtractPictures":"true","Pdf2md":{"type":"docling"}}