Formez PA, the in-house agency of the Department of Public Administration under the Prime Minister’s Office, presented Camilla, the digital assistent that uses artificial intelligence to simplify interactions between citizens and public administrations. Camilla is a CSI product that combines information from public administrations with the potential of generative artificial intelligence.

Camilla is available on the Formez website to help citizens interested in public competitions find useful information. In 2023, more than 400,000 people applied for the notices of competition managed by Formez PA, a potentially very large catchment area also for Camilla.

The Camilla solution

Camilla is the conversational component with generative AI of the artificial intelligence solutions that CSI makes available to its member entities. It combines open-source components and proprietary CSI developments.

The solution consists of the following:

Interface

• a web app for chatting with the assistant, developed with a mobile-first approach and adaptable to all devices;
• a voice interaction, solution, both “voice-to-text” and “text-to-voice”, to interact with the assistant even by speaking;
• a metahuman avatar, realizzato con modelli grafici avanzati, per un’interazione più semplice e naturale. L’avatar è un’interfaccia aggiuntiva, utilizzabile quando Camilla è attivata su totem fisici.

Contenut

a vector database , in which content is collected to be consulted and selected with generative AI engines, using a technique called Retrieval Augmented Generation (RAG). These systems are able to identify the meaning of a question and select relevant content even in the absence of exact semantic matches, thanks to vector similarity techniques (contents with similar meanings are positioned in vectors that are close to each other).

Camilla's Engine

• a back-office configuration and monitoring tool, where CSI or agency operators can customise the assistant's instructions, vary the knowledge domain by inserting single content and collections of content (e.g. website pages) and check the answers given by the assistant, in an aggregate or specific way, to supervise the quality;
• a chain of instructions and mechanisms of interaction between the various components. These elements include functional prompts that enable the assistant to solve intended use cases and avoid unwanted ones, the mechanism for retrieving content from the vector database and conversation context management to allow the assistant to remember messages already exchanged with the user.

Generative AI engine that can be orchestrated across different models and providers based on the needs of the solution developed.

How Camilla works for Formez

Camilla is accessed from the Formez portal, where the notices of competition managed by the entity can be consulted. The interface is a chat window where users can type questions directly, activate the microphone to speak, view the transcript of their requests or select one of the questions suggested by the system.

When it receives a question, Camilla's system:

• identifies whether the question is general or refers to a specific notice;
• searches for the most relevant content based on vector similarity to the question and the previous context of the conversation;
• provides the relevant information and instructions to the generative AI engine;
• returns the response, in either written or spoken form.

If the information Camilla provides is not conclusive, a ticket can be opened. The goal, however, is to gradually reduce such cases.

Camilla's components for Formez are as follows:

The interface
has a chat component, which is suitable for all devices, with a mobile-first approach. Questions can be typed or dictated. Responses are generated by the system and returned in streaming mode, progressively writing the ready-made response parts, without waiting for the end result, to reduce waiting times. Users can enable voice playback of the generated text and provide feedback on the quality of the response (positive or negative). If the resolution allows it, Camilla's avatar, the chat topic and up to three suggested questions to start or continue the conversation will be displayed on the right side of the screen.

Content
Three types of content were used to prepare the responses. The main type consists of the open notices of competition managed by Formez, the texts and metadata of which have been entered into the system to provide specific information, such as the requirements for applying for the competition. All the FAQs provided by the entity are also available, helping potential applicants navigate the most frequently asked questions. Finally, there are some regulatory elements useful for applications. All content was entered into the Camilla system within the vector database hosted by CSI, in which documents are divided into parts (known as “chunks”) and transformed into multidimensional vectors.

Camilla's engine
Camilla's “core” system allows all the other elements to work in harmony. It includes the system's instructions (prompts) which guide the assistant in retrieving information, in both the interpretation and generation phases, maintaining a defined conversational style and avoiding responding on unexpected topics or on which no information is available. There is also a back office tool for adding information and uploading new notices, monitoring the system and the responses provided, and making corrections if necessary.

Generative AI Engine
Generative artificial intelligence is a cutting-edge technological field, where models that guarantee high performance and quality have been developed and made available only by the most advanced technology companies. For Formez, Camilla's generative AI models are provided by Microsoft through the Azure cloud platform. The model currently used is the MS Azure OpenAI gpt-4o, which ensures the best levels of performance, accuracy and security, to develop an intelligent system open to the public.

The monitoring aspect is worth highlighting. Like all generative AI systems, Camilla may provide incorrect information or exhibit unexpected behaviour, since responses are created in real time. The risk of such issues cannot be eliminated, but Formez, CSI, and Microsoft are constantly working to minimise their impact and frequency with various solutions and measures. Therefore, the system of feedback from Camilla users and the constant monitoring of the responses provided are both very important to promptly identify any problems and to improve the system. This monitoring is also important to ensure that Camilla complies with regulations, even if they are not yet in force (e.g. EU AI Act).

People's data and privacy

Camilla's system can be used on a voluntary basis, as an aid in retrieving information on competitions, but it does not replace consulting the text of the notice of competition. People who use Camilla do so by providing their informed consent to the processing of their data. The only personal information that is requested is the user’s voice, and only if they wish to speak to Camilla. This data is processed only for the transcription of the audio in question and is not stored. In general, conversations (question, answer, timestamp and anonymous conversation ID) are tracked anonymously in CSI systems for a limited time, for monitoring purposes, and are accessible by a defined and controlled number of Formez and CSI operators. To ensure compliance with GDPR criteria, the generative AI models provided by Microsoft are hosted on servers located in the European Union and can only be accessed by operators in the European Union. Additionally, no input and output data is made available to other entities or used to improve OpenAI or Microsoft models.