KI certification and security from the perspective of companies

The whitepaper "AI Certification and Security in the Context of the EU AI Act" is available online free of charge at https://doi.org/10.24406/publica-1875

The use of Artificial Intelligence (AI) in private and professional environments is continuously increasing. At the same time, regulations are being discussed at the European level through AI certification and safeguarding measures. The Fraunhofer Institutes IAO and IPA have therefore analyzed the current regulatory measures and summarized the requirements and needs of companies in a new white paper.

In recent years, the importance of Artificial Intelligence (AI) has grown significantly in both private and professional life. AI has the potential to transform numerous industries and areas of our society by improving efficiency and quality in various applications. However, significant risks and uncertainties are also associated with the use of AI, such as algorithmic errors, liability risks, discrimination, and data privacy violations. If AI-based systems are not developed, operated, and tested according to uniform safety standards, they can compromise the safety of products and services.

To address these challenges, the European Union (EU) introduced a draft law for the EU AI Act in 2021, aiming to regulate AI. In June 2023, the EU Commission and the EU Parliament reached an agreement on a proposal and are currently negotiating its implementation with the EU member states. The AI Act, as an important building block for AI certification and safeguarding, is also of central importance to the Innovation Park Artificial Intelligence, which is building the largest AI ecosystem in Europe. »In our Ipai ecosystem, we want to promote AI applications that are not only innovative and efficient but also safe and ethically responsible. A practical understanding of the EU standards from the AI Act for AI safeguarding helps us and our member companies to take all necessary precautions«, emphasizes Moritz Gräter, CEO of Ipai.

In preparation for the upcoming EU AI Act, the Fraunhofer Institute for Industrial Engineering IAO and the Fraunhofer Institute for Production Technology and Automation IPA have examined the perspective of companies. The research team has documented the current state of legal regulations regarding AI safeguarding. Based on the results from interviews with experts, the team also formulated requirements from companies, research, and educational institutions regarding the implementation of safeguarding and certification processes for AI systems.

Lack of guidelines for AI safeguarding as a source of uncertainty for companies

According to the current draft law for the EU AI Act, AI applications are to be classified into different risk levels and subject to varying requirements. Operators of high-risk AI applications will be obliged to verify their conformity with these requirements through a self-assessment and can subsequently use the CE mark as a certificate. Mandatory third-party inspections are only to be required in special areas, such as medical technology.

Although these requirements clearly indicate how a compliant AI deployment with the EU AI Act should look, it is currently unclear how this can be achieved. Concrete measures for verifying compliance are still missing. »Companies, for example, have difficulty assessing under what conditions their AI application is transparent enough or what error rate is tolerable«, explains Janika Kutz, team leader at the Fraunhofer IAO's Research and Innovation Center for Cognitive Service Systems KODIS. Similarly, there is concern that the effort required for certification, especially resources from startups and small to medium-sized enterprises, may be excessive. It is feared that legal expertise will be necessary to fully and correctly implement the requirements, which could increase development times and costs, making it difficult for European companies to compete internationally.

Companies formulate clear requirements for AI certification

An important result of the interviews is that certification must be feasible for companies of all sizes. Securing and certifying every application is labor-intensive and resource-intensive, so companies have clear requirements for regulations regarding AI safeguarding. Factors such as transparency and feasibility of certification processes, clear roles of authorities and institutions, and the preservation of innovation capacity are highlighted as particularly important. »The surveyed companies agree that AI certification should always focus on the added value for end users«, summarizes Prof. Dr. Marco Huber, head of the Cyber Cognitive Intelligence department at Fraunhofer IPA.

External support offers are in demand for companies

Based on the statements of the interviewees, most companies do not seem sufficiently prepared for the upcoming regulations through the EU AI Act. Companies can benefit from information dissemination, knowledge transfer, and network building, and are also interested in individual consulting services as well as practical methods and tools to support safeguarding and certification of AI-based systems. Through ongoing exchange between regulatory authorities, industry, research institutions, and the general public, a comprehensive understanding of the opportunities and challenges of AI use can be developed. Ultimately, this will lead to the development of AI systems that are more effective, safer, and more practical.

The white paper »AI Certification and Safeguarding in the Context of the EU AI Act« is available online free of charge at: https://doi.org/10.24406/publica-1875