​​Your AI Model Is Only as Good as Its Data Flywheel

Generative AI offers the world boundless opportunities for creativity and productivity, but calls for more effective governance are growing louder. The need for AI regulation goes beyond policing misinformation and protecting human expression; user concerns point to a persistent lack of trust. At its core, the challenge of AI integration is making sure organizations are collecting, using, and regenerating the right data. It’s about practicing nuanced controls and applying ethical guidelines to prioritize fair use, accuracy, and integrity. MITRE’s Recommendations for AI Governance MITRE has recently proposed new AI recommendations for the incoming presidential administration, emphasizing the urgency of AI governance. They suggest issuing an executive order mandating AI governance measures within the first six months of the administration’s term. Additionally, MITRE recommends an executive order that mandates system auditability and the development of standards for audit trails. This would include requiring AI developers to disclose the data used to train their systems and the foundation models on which their systems were built. According to MITRE, the ability to audit systems is vital for tracking AI misuse and maintaining public trust in AI. Corporate Efforts in AI Governance Companies like IBM and Salesforce are now allocating resources to internal ethics committees dedicated to AI governance. Lenovo Group, Telefonica, Microsoft, and Mastercard, among others, have also signed a new agreement to build ethical AI by integrating UNESCO’s recommendations. Around the world, organizations and institutions are striving to create trustworthy AI models by conducting regular bias audits and ensuring diverse, representative training data. More than ever, leaders are looking to integrate digital record-keeping solutions like blockchain to gain granular visibility into their data and hold models accountable to emerging ethical standards. Evolving Intellectual Property’s Definition The democratization of AI capabilities has created all kinds of opportunities possible for the average person. Today, anyone can pay a small subscription fee, if any, to use generative AI. Up-and-coming artists can pay $20 per month to leverage Stability AI’s open-source models for creating content and producing music. Developers are able to write code faster with tools like Tabnine and OpenAI’s Codex. Those with disabilities can also access speech-to-text and text-to-speech systems powered by AI to more effectively and easily communicate with others. But with enhancing access to AI comes the risk of misappropriation. Its widespread use has far-reaching implications for privacy and intellectual property, as it blurs the lines of ownership by generating content that may be derived from existing works. Leveraging blockchain technology can address this. Smart contracts – self-executing contracts that trigger actions based on predetermined terms and conditions– automate the licensing of creative works, distinguish AI content from human content with digital tags or unique watermarks, and authenticate the reliability of content by comparing it against a trusted registry of AI models. These contracts improve accountability by maintaining a verifiable record of a work’s lifecycle and ownership. With a decentralized blockchain network, the risk of unauthorized access or modifications could be drastically reduced – and, ultimately, help establish a trusted, transparent ecosystem where data privacy and intellectual property rights are adequately protected. Rooting Out Hallucinations In addition to ensuring integrity and fair use, the democratization of AI has also raised questions about the accuracy of generated outputs. A recent study by startup Vectara shed light on a prevalent issue: the dangerously high rates of hallucination among open-source LLMs available to the public. Even the most reliable models have at least a small chance of fabricating a response to a query. Neuroscientist Olivier Ouillier explained that AI solutions don’t have any intention to deceive or manipulate – they’re simply engineered to predict the most statistically likely strong of words. LLMs compress massive volumes of training data for efficiency, which means the fine details can get lost in their calculations. To fill the gaps, the AI system scrambles to make something up. Blockchain can help companies address this challenge. Using a decentralized record-keeping system, businesses can easily verify the provenance and quality of data being used to train the model and root out inaccuracies that could lead to hallucinations. Today, the Hedera blockchain, which Prove AI is to run on, gives developers the ability to “go back in time” to visit previous iterations of AI systems to understand how their model leaders, operates, and adapts – version control. Integrated within IBM’s watsonx platform, Prove AI will enable real-time monitoring of input and output data for training AI models. It will equip organizations with the power to vet their AI systems, ensure their reliability, uphold ethical standards, and scale their impact with trustworthy data. Documenting a model's entire lifecycle in an immutable ledger – from data acquisition, to training and testing, to setting parameters and optimizing refinement – empowers users to gain a clearer understanding of how their AI systems reached specific conclusions. To learn more on how Prove AI, visit our Prove AI landing page or contact us directly.

We’re excited to announce Prove AI’s participation at IBM TechXchange, happening October 21-24, 2024 in Las Vegas. As a critical piece of the AI governance puzzle, Prove AI is a comprehensive AI governance solution that enables certifiable and tamper-proof auditing for organizations building, training and running AI models. By bringing unprecedented visibility and trust to AI solutions, organizations can embrace AI with confidence and comply with all relevant regulatory requirements. Prove AI will take center stage with sessions led by our CEO, Mrinal Manohar, and VP of Product Management, Henry Guo. Highlights include the keynote “Shaping the Future of AI Governance” with Mrinal Manohar, a technical deep dive and live demo led by Henry Guo and Brian Green, SVP of AI Solutions at Greyscale AI, and a sandbox session with our own CTO, Greg Whalen. See how Prove AI is setting a new standard for AI governance. The event and presentations will also be streamed online and recorded by our team. Here’s what to expect from Prove AI at TechXchange: Keynote: Shaping the Future of AI Governance Tuesday, October 22 | 11:30 AM PT Our CEO, Mrinal Manohar, will join IBM’s GM for Data & AI, Rita Gunnar, on stage to discuss Prove AI’s pivotal role in AI governance alongside our partner, Greyscale AI. Together, they will explore how Prove AI enables companies to securely manage AI data across multiple parties, maintaining a tamper-proof, digital record of all activity. This functionality is crucial for organizations navigating the EU AI Act, NIST guidelines, and other upcoming regulations and guidelines. Attendees will learn how Prove AI’s technology helps companies establish both provenance and access, ensuring compliance with evolving AI governance standards. Sandbox Session: Ask the Experts on AI Governance Tuesday, October 22 | 3:30 PM PT In an interactive AMA session, our CTO, Greg Whalen, will be on hand to answer questions about AI governance and the practical applications of Prove AI. As AI regulations tighten, companies must be able to ensure accountability and compliance with frameworks like the EU AI Act, NIST and more. Greg will discuss how Prove AI automates critical functions like tracking AI risk levels and ensuring that AI systems comply with governance frameworks, making it easier for companies to prove compliance in real-time without relying on outdated manual processes. Technical Deep Dive: Live Demo Tuesday, October 22 | 4:30 PM PT Join Henry Guo, our VP of Product Management, and Brian Green, SVP of AI Solutions at Greyscale AI, for a deep dive into Prove AI’s technical capabilities. This session will include a live demo, showcasing how Prove AI allows companies to govern multi-party access, maintain tamper-proof recording using blockchain technology, and more. With Prove AI, organizations can store comprehensive, fully auditable logs of AI training data, ensuring that any changes – whether internal or third-party – are tracked in a transparent and secure manner. This demo will provide an in-depth look at how Prove AI’s automated compliance features streamline the process of meeting regulatory mandates, reducing manual efforts like spreadsheets, lists, and emails. Meet Us at Booth #235: Preview Prove AI Throughout the event, we invite developers and attendees to stop by our booth for an exclusive preview of Prove AI. Discover how our platform’s automated compliance features eliminate the need for manual governance processes. With Prove AI, companies can efficiently monitor AI system risks, govern access to sensitive AI data, and maintain the highest standards of accountability. This is your change to see how Prove AI’s solutions can transform the way your organization manages AI governance, helping you meet regulatory demands. Apply for early access for Prove AI!

Prove AI CTO Greg Whalen spoke at the NY AI Summit, one of the world’s largest annual gatherings of AI professionals, earlier this week. His presentation highlighted a new paradigm that has emerged alongside AI: AI development does not follow the same rules as traditional software development. Unlike previous technologies, the data, events and software generated (and consumed) by AI are intertwined and operate at a significantly larger and more dynamic scale. This shift is particularly relevant as organizations increasingly prioritize clear and comprehensive AI governance strategies. For governance to be truly effective, organizations need the ability to observe all inputs that train a given AI system. This represents a stark departure from existing norms. As AI standards gain traction – the most notable example being NIST’s ISO 42001 standard, which was officially achieved by AWS last month – it has become increasingly clear that there’s a direct correlation between AI-related risk levels and a lack of observability into AI systems’ underlying datasets and models. Prove AI was purpose-built for this new AI development paradigm. Using distributed ledger technology as a highly secure and tamper-proof infrastructure layer, it enables organizations to capture all key AI data and automate the management.This ensures a fully observable and ISO 42001-compliant AI governance process. During his presentation, Greg shared several compelling examples that highlight the importance of continuous AI observability and governance: 1. Always Compliant and Observable Development Greg emphasized the importance of maintaining compliance and observability throughout AI development, no matter the tools or teams involved. For instance, a company with 50 AI teams using diverse MLOps tools can implement a system that logs every action in real time, avoiding reliance on periodic reviews or single-tool mandates. This approach ensures compliance is seamless, transparent, and integrated into the development process without adding manual work or delays. 2. Industrial AI and Public Safety Greg addressed the risks industries face when embedding AI into physical products like robotic devices or x-ray machines. Automating safety and maintenance reviews can enhance accountability while minimizing risks and manual effort. Transparent, tamper-proof records of incremental updates help manufacturers share responsibility with stakeholders and prevent liability issues or legal disputes. 3. Third-Party Compliance and Verifiability For software vendors, traditional audits and certifications can be time-consuming and complex, especially when working with LLMs or third-party tools. Greg proposed a secure, real-time “show your work” system that eliminates the need for manual verifications, reducing friction and ensuring continuous compliance. This approach provides stakeholders with instant access to tamper-proof compliance records, fostering trust and streamlining the process. While this is far from an exhaustive list of Prove AI’s applications, it does provide a powerful testament to the efficiencies driven by good AI governance. If you’re beginning your AI governance journey, we’d love to hear from you and explore how we can help set you up for success. Get in touch with us here.