Today, we’re excited to announce SafeCoder, a code assistant solution built for enterprises.
SafeCoder’s goal is to unlock enterprise software development productivity using a fully compliant, self-hosted pair of programmers. In marketing, it’s your own on-premises GitHub copilot.
Before we go into more detail, here’s what you need to know:
SafeCoder is not a model, but a complete end-to-end commercial solution. SafeCoder is built with security and privacy as core principles. Your code never leaves your VPC during training or inference. SafeCoder is designed for customers to self-host on their own infrastructure. SafeCoder is designed to allow customers to own their own code large-scale language models.
Why Safecoder?
Code assistance solutions built on LLM, such as GitHub Copilot, have significantly increased productivity. For companies, the ability to tailor Code LLMs with the company’s code base to create their own Code LLMs increases the reliability and relevance of completions, providing another level of productivity. For example, Google’s internal LLM code assistant reports completion approval rates of 25-34% when trained on an internal code base.
However, relying on the closed-source Code LLM to create internal code assistants exposes companies to compliance and security issues. Fine-tuning the closed-source Code LLM with an internal codebase requires exposing this codebase to third parties, which initially occurs during training. And fine-tuned code LLMs can “leak” code from the training dataset during inference. To meet compliance requirements, companies must deploy a fine-tuned Code LLM within their infrastructure, which is not possible with a closed source LLM.
With SafeCoder, Hugging Face allows customers to build their own fine-tuned code LLMs in their own codebase using state-of-the-art open models and libraries without having to share the code with Hugging Face or other third parties. Hugging Face uses SafeCoder to provide a containerized, hardware-accelerated code LLM inference solution. It can be deployed directly within a customer’s secure infrastructure without the customer having to type and type code from their secure IT environment.
From StarCoder to SafeCoder
At the core of the SafeCoder solution is Code LLM’s StarCoder family, created by the BigCode project, a collaboration between Hugging Face, ServiceNow, and the open source community.
StarCoder models offer unique characteristics that make them ideal for enterprise self-hosted solutions.
State-of-the-art code completion results – see paper benchmarks and multilingual code evaluation leaderboards Designed for inference performance: 15B parameter model with code optimizations, multi-query attention to reduce memory footprint, and flash attention to scale to 8,192 token contexts. Trained on Stack, an ethically sourced open source code dataset containing only commercially licensed code, it has developer opt-out mechanisms in place from the start and has been refined through intensive PII removal and deduplication efforts.
Note: While StarCoder is the inspiration and model that powers the initial version of SafeCoder, a key benefit of building an LLM solution on an open source model is the ability to adapt to the latest and greatest open source models available. In the future, SafeCoder may offer other similarly commercially acceptable open source models built on transparent, ethically sourced datasets as a base LLM for fine-tuning.
Privacy and security as fundamental principles
For any company, its internal codebase is some of its most important and valuable intellectual property. A core principle of SafeCoder is that a customer’s internal codebase is never accessed by third parties (including Hugging Face) during training or inference.
During the initial setup phase of SafeCoder, the Hugging Face team will provide containers, scripts, and samples to work with the customer to select, extract, prepare, replicate, and anonymize internal codebase data into training datasets used in training containers provided by Hugging Face configured on the customer’s available hardware infrastructure.
During the SafeCoder deployment phase, customers deploy containers provided by Hugging Face onto their infrastructure and expose internal private endpoints within their VPC. These containers are configured for the exact hardware configurations available to you, such as NVIDIA GPUs, AMD Instinct GPUs, Intel Xeon CPUs, AWS Inferentia2 or Habana Gaudi accelerators.
Compliance as a fundamental principle
Regulatory frameworks for machine learning models and datasets are still being developed around the world, so global companies need to ensure that the solutions they use minimize legal risk.
Data sources, data governance, and managing copyrighted data are just some of the most important compliance areas to consider. BigScience, the original and inspiration for BigCode, worked in working groups on these areas before they were widely recognized in the draft AI EU law, and as a result was rated the most compliant of foundational model providers in the Stanford CRFM study.
BigCode has expanded on this work by implementing new technologies in the code domain and building a stack with compliance as a core principle, including commercially acceptable license filtering, consent mechanisms (allowing developers to easily see if their code exists and request to opt-out of datasets), extensive documentation and tools for inspecting source data, and dataset improvements (such as deduplication and PII removal).
All these efforts lead to the minimization of legal risks for users of StarCoder models and SafeCoder customers. And for SafeCoder users, these efforts translate into compliance capabilities. When software developers get code completion, these suggestions are checked against the stack, so the user knows whether the suggested code matches existing code in the source dataset and what the license is. Customers can specify which licenses they prefer and present those preferences to users.
How does it work?
SafeCoder is a complete commercial solution including services, software and support.
Train your own SafeCoder model
StarCoder is trained on over 80 programming languages and provides state-of-the-art performance on multiple benchmarks. We begin our efforts with an optional training phase to provide better code suggestions, especially for SafeCoder customers. During this phase, the Hugging Face team works directly with customer teams to guide them through the preparation and construction of training code datasets and through fine-tuning to create their own code generation models, without exposing the codebase to third parties or the internet.
The end result is a model that fits the customer’s coding language, standards, and practices. Through this process, SafeCoder customers learn processes, build pipelines to create and update their own models, avoid vendor lock-in, and maintain control over their AI capabilities.
Introducing SafeCoder
During the setup phase, SafeCoder customers and Hugging Face design and provision the optimal infrastructure to support the concurrency necessary to provide a great developer experience. Hugging Face then builds a hardware-accelerated, throughput-optimized SafeCoder inference container that customers can deploy on their own infrastructure.
SafeCoder inference supports a variety of hardware, giving customers a wide range of options, including NVIDIA Ampere GPUs, AMD Instinct GPUs, Habana Gaudi2, AWS Inferentia 2, and Intel Xeon Sapphire Rapids CPUs.
Using SafeCoder
Once SafeCoder is deployed and its endpoints are running within a customer VPC, developers can install compatible SafeCoder IDE plugins to get code suggestions while they are running. SafeCoder currently supports popular IDEs such as VSCode, IntelliJ, and other plugins from partners.
How do I get SafeCoder?
Today at the VMware Explore conference, we are working with VMware to announce SafeCoder and make it available to VMware enterprise customers. By working with VMware, we can ensure a successful deployment of SafeCoder into your VMware Cloud infrastructure, whether it’s your preferred cloud, on-premises, or hybrid infrastructure scenario. In addition to leveraging SafeCoder, VMware has published a reference architecture that includes code samples to help you deploy and operate SafeCoder on your VMware infrastructure to minimize time to value. VMware’s Private AI Reference Architecture makes it easy for organizations to quickly leverage popular open source projects like ray and kubeflow to deploy AI services adjacent to private datasets. We’re also working with Hugging Face to help organizations maintain the flexibility to leverage the latest and greatest open source models. All of this is achieved without sacrificing total cost of ownership or performance.
“Our collaboration with Hugging Face around SafeCoder aligns perfectly with VMware’s goal of empowering customers to choose their solution while maintaining privacy and control over their business data. In fact, we’ve been running SafeCoder internally for months with excellent results. Best of all, our collaboration with Hugging Face is just beginning, and we’re excited to bring our solution to hundreds of thousands of customers around the world.” said Chris Wolf, vice president of Labs. Learn more about private AI and how VMware differentiates itself in this emerging space.
If your company is interested in SafeCoder, please contact us here. Our team will contact you to discuss your requirements.
