Why Explainability Begins Long Before Model Training

Generative AI offers the world boundless opportunities for creativity and productivity, but calls for more effective governance are growing louder. The need for AI regulation goes beyond policing misinformation and protecting human expression; user concerns point to a persistent lack of trust. At its core, the challenge of AI integration is making sure organizations are collecting, using, and regenerating the right data. It’s about practicing nuanced controls and applying ethical guidelines to prioritize fair use, accuracy, and integrity. MITRE’s Recommendations for AI Governance MITRE has recently proposed new AI recommendations for the incoming presidential administration, emphasizing the urgency of AI governance. They suggest issuing an executive order mandating AI governance measures within the first six months of the administration’s term. Additionally, MITRE recommends an executive order that mandates system auditability and the development of standards for audit trails. This would include requiring AI developers to disclose the data used to train their systems and the foundation models on which their systems were built. According to MITRE, the ability to audit systems is vital for tracking AI misuse and maintaining public trust in AI. Corporate Efforts in AI Governance Companies like IBM and Salesforce are now allocating resources to internal ethics committees dedicated to AI governance. Lenovo Group, Telefonica, Microsoft, and Mastercard, among others, have also signed a new agreement to build ethical AI by integrating UNESCO’s recommendations. Around the world, organizations and institutions are striving to create trustworthy AI models by conducting regular bias audits and ensuring diverse, representative training data. More than ever, leaders are looking to integrate digital record-keeping solutions like blockchain to gain granular visibility into their data and hold models accountable to emerging ethical standards. Evolving Intellectual Property’s Definition The democratization of AI capabilities has created all kinds of opportunities possible for the average person. Today, anyone can pay a small subscription fee, if any, to use generative AI. Up-and-coming artists can pay $20 per month to leverage Stability AI’s open-source models for creating content and producing music. Developers are able to write code faster with tools like Tabnine and OpenAI’s Codex. Those with disabilities can also access speech-to-text and text-to-speech systems powered by AI to more effectively and easily communicate with others. But with enhancing access to AI comes the risk of misappropriation. Its widespread use has far-reaching implications for privacy and intellectual property, as it blurs the lines of ownership by generating content that may be derived from existing works. Leveraging blockchain technology can address this. Smart contracts – self-executing contracts that trigger actions based on predetermined terms and conditions– automate the licensing of creative works, distinguish AI content from human content with digital tags or unique watermarks, and authenticate the reliability of content by comparing it against a trusted registry of AI models. These contracts improve accountability by maintaining a verifiable record of a work’s lifecycle and ownership. With a decentralized blockchain network, the risk of unauthorized access or modifications could be drastically reduced – and, ultimately, help establish a trusted, transparent ecosystem where data privacy and intellectual property rights are adequately protected. Rooting Out Hallucinations In addition to ensuring integrity and fair use, the democratization of AI has also raised questions about the accuracy of generated outputs. A recent study by startup Vectara shed light on a prevalent issue: the dangerously high rates of hallucination among open-source LLMs available to the public. Even the most reliable models have at least a small chance of fabricating a response to a query. Neuroscientist Olivier Ouillier explained that AI solutions don’t have any intention to deceive or manipulate – they’re simply engineered to predict the most statistically likely strong of words. LLMs compress massive volumes of training data for efficiency, which means the fine details can get lost in their calculations. To fill the gaps, the AI system scrambles to make something up. Blockchain can help companies address this challenge. Using a decentralized record-keeping system, businesses can easily verify the provenance and quality of data being used to train the model and root out inaccuracies that could lead to hallucinations. Today, the Hedera blockchain, which Prove AI is to run on, gives developers the ability to “go back in time” to visit previous iterations of AI systems to understand how their model leaders, operates, and adapts – version control. Integrated within IBM’s watsonx platform, Prove AI will enable real-time monitoring of input and output data for training AI models. It will equip organizations with the power to vet their AI systems, ensure their reliability, uphold ethical standards, and scale their impact with trustworthy data. Documenting a model's entire lifecycle in an immutable ledger – from data acquisition, to training and testing, to setting parameters and optimizing refinement – empowers users to gain a clearer understanding of how their AI systems reached specific conclusions. To learn more on how Prove AI, visit our Prove AI landing page or contact us directly.

As AI becomes a central part of business operations, the risks tied to its use are more pressing than ever. Data breaches, biased outcomes, and AI systems behaving in unexpected ways can undermine trust and disrupt businesses. In response, Gartner has developed the TRiSM (Trust, Risk, and Security Management) framework, a comprehensive approach to managing AI governance and security. Its recent spotlight, “Tackling Trust, Risk and Security in AI Models” included Prove AI as a featured solution, alongside other AI industry leaders. With AI risks becoming more visible, companies need clear strategies to protect their systems. Gartner’s TRiSM framework offers a roadmap for businesses to secure their AI. The Basics of TRiSM Gartner’s TRiSM framework focuses on dealing with two big risks in AI – data breaches and AI behaving unpredictably. The solution is managing these risks through three main areas: people, processes, and technology. People: AI governance should be a team effort. Companies need dedicated AI teams to ensure there’s accountability. Processes: Balancing risks versus rewards is key. As regulations like the EU AI Act emerge, companies must ensure their AI governance is strong enough to reduce threats. Technology: The right tools are essential to handle trust, risk, and security in AI. TRiSM covers the entire AI life cycle, from data protection to model integrity. How Prove AI Simplifies AI Governance Prove AI was built to make responsible AI governance more efficient and predictable for businesses. It gives organizations tamper-proof oversight of their AI data, models, and outputs – keeping everything secure and accountable. Powered by blockchain technology, Prove AI ensures every part of your AI process is locked down, from training data to model performance. Our solution aligns perfectly with TRiSM, helping companies maintain trust, transparency, and security in their AI systems. AI can be a game-changer, but only if it’s managed properly. Gartner’s TRiSM framework is a key resource for organizations seeking to keep their AI systems safe and compliant. By getting ahead of these risks, businesses can make sure their AI is as secure and responsible as it needs to be. To read more about Gartner’s recent TRiSM findings please visit their website here. Additionally, please reach out directly if you’re interested in becoming one of the first users of Prove AI or speaking with one of our team members.

Prove AI CTO Greg Whalen spoke at the NY AI Summit, one of the world’s largest annual gatherings of AI professionals, earlier this week. His presentation highlighted a new paradigm that has emerged alongside AI: AI development does not follow the same rules as traditional software development. Unlike previous technologies, the data, events and software generated (and consumed) by AI are intertwined and operate at a significantly larger and more dynamic scale. This shift is particularly relevant as organizations increasingly prioritize clear and comprehensive AI governance strategies. For governance to be truly effective, organizations need the ability to observe all inputs that train a given AI system. This represents a stark departure from existing norms. As AI standards gain traction – the most notable example being NIST’s ISO 42001 standard, which was officially achieved by AWS last month – it has become increasingly clear that there’s a direct correlation between AI-related risk levels and a lack of observability into AI systems’ underlying datasets and models. Prove AI was purpose-built for this new AI development paradigm. Using distributed ledger technology as a highly secure and tamper-proof infrastructure layer, it enables organizations to capture all key AI data and automate the management.This ensures a fully observable and ISO 42001-compliant AI governance process. During his presentation, Greg shared several compelling examples that highlight the importance of continuous AI observability and governance: 1. Always Compliant and Observable Development Greg emphasized the importance of maintaining compliance and observability throughout AI development, no matter the tools or teams involved. For instance, a company with 50 AI teams using diverse MLOps tools can implement a system that logs every action in real time, avoiding reliance on periodic reviews or single-tool mandates. This approach ensures compliance is seamless, transparent, and integrated into the development process without adding manual work or delays. 2. Industrial AI and Public Safety Greg addressed the risks industries face when embedding AI into physical products like robotic devices or x-ray machines. Automating safety and maintenance reviews can enhance accountability while minimizing risks and manual effort. Transparent, tamper-proof records of incremental updates help manufacturers share responsibility with stakeholders and prevent liability issues or legal disputes. 3. Third-Party Compliance and Verifiability For software vendors, traditional audits and certifications can be time-consuming and complex, especially when working with LLMs or third-party tools. Greg proposed a secure, real-time “show your work” system that eliminates the need for manual verifications, reducing friction and ensuring continuous compliance. This approach provides stakeholders with instant access to tamper-proof compliance records, fostering trust and streamlining the process. While this is far from an exhaustive list of Prove AI’s applications, it does provide a powerful testament to the efficiencies driven by good AI governance. If you’re beginning your AI governance journey, we’d love to hear from you and explore how we can help set you up for success. Get in touch with us here.