Adoption Approaches
Legislative bodies aiming to harness the benefits of AI have myriad options for deploying AI-enabled tools, each presenting a unique blend of control, customization, and institutional integration. These options range from using existing commercial tools — many of which are cloud services such as Google Workspace, Microsoft365, Salesforce, Airtable, Mailchimp, etc. — that are incorporating AI into their core offerings. Another option is the current route taken by the US House and Senate, establishing policies for use of commercial AI products such as OpenAI’s GPTPlus, Google’s Bard, and Anthropic’s Claude. As discussed above, however, these subscription services should only be used with non-sensitive, public information, which limits their utility for legislative staff. The next level of adoption, as with GAO’s Galileo project, is for legislatures to leverage third party models (such as OpenAI or Anthropic Claude) through an advanced programming interface (API), to query and derive insights from their own internal data. Current use terms for these companies stipulate that data passed through APIs is not retained or used to train their underlying model, allowing for greater security for legislative queries and applications. Over the long term, however, especially as the technology evolves and new methods for fine-tuning open source models with proprietary data emerge, legislatures may be able to create and maintain their own public sector LLMs. However, this final option is likely too resource intensive with the current state of the technology.
COTS: Commercial Off-the-Shelf software that leverages AI
3rd-Party GenAI: Using third-party GenAI tools for basic tasks
Custom GenAI: Developing custom apps (either institutional or with contractor) that leverage commercial models
Institutional LLM: Maintaining institutional LLM, potentially leveraging open source models
Third Party COTS (Commercial Off-the-Shelf) Applications
Third Party applications — such as Microsoft365, Google Workspace, Canvas, Airtable, and other cloud services — provide AI-enabled services through services that legislative staff may already be using in their workflows. These tools enable legislative staff to easily integrate new capabilities without upfront development costs or even new procurement. While these tools will necessarily be more general and not specific to the legislative workflow, they may provide significant efficiencies for non-legislative functions such as document drafting, information summarization, and scheduling.
Commercial GenAI Applications
Legislative staff are already experimenting with commercial GenAI tools like ChatGPT/GPT-4, Claude, and Bard to assist with writing, research, and constituent interactions. While these tools can provide helpful drafts and suggestions, their use requires diligent verification to protect against factual errors, bias, and potential copyright issues. In addition, the terms of service for use of the basic interface with these models in most cases allows any information entered to be used to “train” the underlying model. As noted above, both the US House and Senate have approved the use of some of these tools for research and experimentation, with warnings against entering personally identifiable or non-public information. These privacy and data concerns will limit the usefulness of the commercial tools for further integration into legislative workflows, highlighting the need for government entities to make custom tools available in the future.
Custom GenAI Tools that Leverage Third Party Models
In order to address some of the privacy and data quality concerns with using the public interface for commercial models, institutions can develop their own interface that leverages “retrieval-augmented generation” (RAG) to improve the quality of responses by grounding the model on institutional data. (This approach will be discussed in detail in volume 2.) GAO uses this approach for its “Project Galileo” — leveraging a commercial API to return responses that reference GAO’s internal data.¹ This approach allows higher quality, more relevant responses, and queries are not used to train the underlying commercial model. However, the use of this method requires significant in-house technological skill, organized data management, and a commitment to ongoing maintenance and support for the technology.
Institutional Large Language Models
In the long term, it is possible that legislatures will develop and support their own institutional LLMs, trained on legislative text and the vast corpus of precedents, statements, testimony, and other records of the institution. This technology is still so new that it is not yet clear what the best or most efficient architectural approach will be for large organizations and enterprises as they approach this question. With current capabilities, developing an institutional model from scratch would likely be too resource-intensive, requiring massive datasets, extensive compute power, machine learning expertise, and a long-term investment. However, as open source models become more capable and similar institutions experiment with different approaches, bespoke institutional models may become a more viable path.
¹ Cate Burgan, “GAO Building GenAI Tools to Face Internal Challenges,” MeriTalk (November 9, 2023)
