Local Open AI Models: Public Infrastructure Instead of Digital Dependency

Artificial intelligence is entering public administration, healthcare, education, local government and state security services. The central question is not whether public institutions will use AI. They already will. The real question is who will control the infrastructure, the data, the models, the logs and the rules of use. Public authorities can either build internal capacity through open-source tools, auditable systems and domestic expertise, or they can outsource critical functions to closed cloud platforms, priced per token and governed by terms that may change unilaterally.

For municipalities, regions, universities, hospitals, security agencies and ministries, the most appropriate starting point is low-cost local AI based on open-source software and, wherever possible, open models. This means locally controlled inference, retrieval-augmented generation over trusted institutional sources, full logging, human accountability, model documentation and independent auditability. This is not a romantic preference for open source. It is a practical strategy for democratic sovereignty, fiscal discipline and administrative capability.

Cost matters because experimentation must precede procurement

The public debate often presents AI as if every useful application requires hyperscale data centres and billion-euro investments. That is true for training the largest frontier foundation models. It is not true for most public-sector use cases. A very large share of public value comes from document search, summarisation, classification, transcription, anonymisation, structured data extraction, internal helpdesks, procurement checks, service triage and legal or administrative retrieval over official documents.

A small public organisation can start with a modest local testbed, validate concrete use cases and scale only the services that prove useful. This “test before invest” logic is essential. A local lab built with affordable hardware, commodity GPUs, Ollama, llama.cpp, vLLM, PostgreSQL, pgvector, Qdrant, Keycloak, Prometheus and Grafana can demonstrate in weeks whether an application is accurate, useful and safe enough for further investment. This reverses the usual procurement failure: buying large proprietary systems before the administration understands its real needs.

Low-cost local AI also changes the economics of the state. Instead of paying indefinite rent to external providers, public institutions invest in reusable infrastructure. Instead of every municipality, university or hospital buying separate closed tools, they can share a common stack, common deployment patterns, common evaluation methods and common documentation. The result is not only lower cost. It is institutional learning.

Municipalities and regions: AI close to citizens

Municipalities need AI for practical, everyday problems. A citizen service centre can use Retrieval-Augmented Generation(RAG) over official procedures, local regulations and municipal forms to inform citizens about required documents. A municipal contact line can classify requests about waste collection, lighting, road damage, social support or permits. Technical departments can summarise project files, detect missing documentation, compare procurement specifications and identify formats that create unjustified vendor lock-in.

Regions need a broader layer of capacity. Civil protection, environmental monitoring, transport planning, social policy and agricultural support require geospatial data, satellite imagery, administrative records and historical incident data. A regional AI cluster can support climate risk maps, flood-risk analysis, social vulnerability dashboards, environmental compliance checks and logistics during emergencies. The key design principle is data proximity. Data produced in a region and affecting a regional population should not automatically leave the institutional environment that is responsible for it.

Universities and hospitals: research capacity with data protection

Universities are natural hosts for local open AI infrastructure. They can evaluate models, build Greek language corpora, fine-tune domain models, train students and civil servants, and publish reusable documentation. Their role should not be limited to consultancy. Universities should help create public digital assets: code, evaluation datasets, reproducible pipelines, model cards, datasheets and open training material.

Hospitals require an even stricter approach. A hospital LLM should not diagnose patients or prescribe treatment. It can support safer administrative and knowledge workflows: searching clinical guidelines, summarising internal documents, transcribing meetings, anonymising clinical texts, drafting patient information under medical supervision, and assisting staff with internal procedures. International experience in health data analysis shows that trust comes from secure environments, open-source tooling, pseudonymisation, access control and audit trails, not from sending sensitive data to opaque external systems.

Ministries and security agencies: sovereignty without automated power

For ministries, local LLMs can support legal research, public consultation analysis, procurement review, tax guidance, policy brief preparation, document classification and programme evaluation. But the governance rule must be explicit: AI may assist, retrieve, summarise and explain. It must not issue administrative decisions.

For security-related public services, the strongest use cases are cybersecurity, incident analysis, open-source intelligence processing, technical knowledge retrieval, and secure internal reporting. Local deployment is particularly important here. No responsible public authority should send sensitive security data to closed external APIs. At the same time, safeguards must be strict: no mass surveillance by default, no automated targeting of citizens, no use without a legal basis, no operational deployment without human oversight, logs and accountability.

International practice already points in this direction

France’s Albert, Estonia’s Bürokratt, Switzerland’s Apertus, Allen AI’s OLMo, the European AI Factories and OpenSAFELY in healthcare all point to the same strategic direction. Public-interest AI needs openness, data sovereignty, transparency, human oversight and institutional accountability. Greece does not have to start from scratch. It already has universities, research institutes, open-source communities, legal and administrative data, local government needs, public-sector innovation hubs and European funding instruments.

The political choice is simple. Public institutions can become permanent tenants of proprietary AI systems, or they can become co-owners of shared public AI infrastructure. Low-cost local open-source AI models are the most realistic route to the second option. They allow small starts, rapid validation, local adaptation, data protection, reuse across institutions and democratic control. In public administration, the best AI system is not the most spectacular one. It is the one that public institutions can understand, govern, audit, maintain and improve.

Sources:

Mistral AI, Mistral 7B & Mixtral Research: Mistral AI is an important European example of the development of powerful open-weight models, with Mistral 7B released under the Apache 2.0 license and Mixtral 8x7B available as an open-weights model suitable for efficient local or sovereign deployment: https://mistral.ai/news/announcing-mistral-7b/ and https://mistral.ai/news/mixtral-of-experts/,

Allen Institute for AI, OLMo: OLMo is one of the most important examples of a fully open approach to language models, with the publication of model weights, code and technical reports, making it a useful reference model for publicly auditable AI: https://allenai.org/olmo,

Swiss AI Initiative, Apertus: Apertus, developed by EPFL, ETH Zurich and CSCS, is a fully open, transparent and multilingual LLM, designed as foundational infrastructure for sovereign and trustworthy AI in research, society and industry: https://www.swiss-ai.org/apertus and https://ethz.ch/en/news-and-events/eth-news/news/2025/09/press-release-apertus-a-fully-open-transparent-multilingual-language-model.html,

DINUM / French Government, Albert: Albert is a French example of sovereign public AI for civil servants and frontline public services, based on open models and adapted to the needs of public administration: https://www.info.gouv.fr/actualite/ia-connaissez-vous-albert and https://www.cnil.fr/en/artificial-intelligence-and-public-services-sandbox-cnil-supports-8-innovative-projects,

Estonia, Bürokratt: Bürokratt is an international good practice example of an interoperable network of public digital assistants, with an open development model and the ability to provide information and services through multiple public bodies: https://www.kratid.ee/en/burokratt and https://interoperable-europe.ec.europa.eu/collection/open-source-observatory-osor/government-virtual-assistant-burokratt,

OpenSAFELY, University of Oxford and NHS England: OpenSAFELY demonstrates how sensitive health data can be analysed through an open, secure and controlled computational approach, without uncontrolled transfer of data outside a protected environment: https://www.opensafely.org/ and https://www.phc.ox.ac.uk/research/covid-19/projects/opensafely-a-new-secure-analytics-platform-for-electronic-health-records,

European Commission / EuroHPC and AI Factories: European AI Factories use EuroHPC computing capacity for trustworthy generative models and are planned as hubs for academia, research, the public sector and businesses, with a focus on health, language, culture and sustainability: https://digital-strategy.ec.europa.eu/en/policies/ai-factories and https://www.eurohpc-ju.europa.eu/ai-factories/greece_en,

European Commission, EU AI Act: The European framework for AI makes technical and organisational compliance necessary, especially for high-risk systems, including risk management, transparency, data quality, human oversight, accuracy, robustness and cybersecurity: https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai.