ConWeb is a framework enabling Web Navigation by voice (https://hintlab.polimi.it/projects/). The framework leverages conversational AI and LLMs to build an inclusive interaction paradigm that allows people with different disabilities and different situational needs to interact with web pages in a way that does not depend on mouse interaction and on content’s visual presentation. Currentky, the framework effectively supports reading and navigation tasks. 

This project aims to understand how LLMs can be used to extend ConWeb for:

• Handling long transactions that require negotiation between the AI system and the user conversationally
• Perform complex actions on the web, such as signing in or filling out forms, securely.

The student undertaking this project will be required to:

• Understand the user privacy implications that may arise when interacting conversationally through voice interaction to transmit sensitive information.
• Understand how to build user trust in the system when negotiation with a Conversational Agent happens to gather sensible data.