In our last post (From Scratch to GPU), we introduced the 🤗 Kernel project, which aims to standardize the packaging, distribution, and usage of custom kernels. We want the project to be as hub-friendly as possible while being smooth and secure.
Over the past few months, we have been working towards this goal. In the process, we almost completely redesigned the project. This post summarizes the major updates released so far and what’s coming next.
table of contents
Kernel – new repository type
We’ve introduced a new repository type to the hub called “Kernel”. This allows us to accommodate users with computing-related specificities. For example, users can understand which accelerators, operating systems, and backend versions are supported by a particular kernel.
All available kernels on the hub can be viewed at https://huggingface.co/kernels.
Making these kernels first-class citizens of the hub also benefits the AI ecosystem. Users can now see trends across kernels, models, and applications that use them. Makes it easier for users to find the kernel.
Improved security
Because the kernel executes native code with the same privileges as the Python process that loads the native code, a malicious kernel can cause real harm. Therefore, security is always of paramount importance for kernel projects.
This is why we focused on reproducibility early on. You should be able to recompile the kernel yourself and verify that it matches the published sources. We use Nix to make this possible. This is to keep builds pure through sealed evaluation of build recipes and a strongly isolated sandbox. We further improve provenance by embedding source Git SHA1 into the kernel itself.
Over the past few months, we’ve added additional layers of defense: trusted kernel publishers and code signing.
Trusted kernel publisher
The new repository type also introduces ‘Trusted Publishers’. Because the kernel executes code on the machine with the same privileges as the Python process being used, an attacker could compromise the machine by uploading a malicious kernel and tricking it into using that kernel. To help avoid these malicious kernels, kernel packages now only load kernels from trusted publishers by default. A trusted publisher is an organization that is trusted by the community to act with integrity.
We would like to continue to support loading kernels from organizations or users who are not trusted publishers, but you must explicitly opt in using the trust_remote_code argument when loading kernels from your hub.
from kernel import get_kernel kernel_module = get_kernel( “Atlas-Inference/gdn”, version=1trust_remote_code=truth
)
By default, users cannot publish kernel repositories on the hub. They need to request to be a kernel publisher. Users and organizations can request access through their account settings. This allows us time to process these requests on a case-by-case basis.
kernel signature
The security layer we’re adding is code signing. Code signing protects against scenarios where an attacker uploads a malicious kernel to a kernel repository from a trusted publisher whose hub credentials have been compromised. Code signing involves signing the kernel using a private key known only to the kernel developer and verifying it using a publicly available public key. In a hub compromise scenario, the attacker is unable to sign the malicious kernel because he does not possess the private key required for signing.
To further improve security, use Sigstore Signing to sign using a temporary private key. These signing keys are only valid for a limited time, so even if the private key is compromised, an attacker usually cannot use it. It also verifies that the kernel is signed by a trusted GitHub workflow from a trusted GitHub repository.
Kernel signatures are already supported in kernel-builder, which provides kernel verify-signature to verify kernels. The kernel does not yet verify the signature when the kernel loads because we want to further test this new feature before fully deploying it. Preliminary notes about configuring code signing for your own kernel can be found in the kernel 0.16.0 release notes: https://github.com/huggingface/kernels/releases/tag/v0.16.0.
Improved CLI
Previously, a number of utilities were intertwined between the kernel and the kernel builder. We have established a better separation of concerns between the kernel CLI and the kernel builder. The mental model here is that the kernel is a library that loads and prepares the kernel for use. Therefore, it should not contain anything related to “building” the kernel.
As a result, both the kernel and the kernel builder are leaner and more specific. Please see the documentation for more information on this.
More coverage of frameworks and backends {#more-coverage-of-frameworks-and-backends}
Extended framework support. The most visible changes are:
Added support for Torch Stable ABI to the kernel and kernel builder. The Torchlamp Stable ABI allows kernel developers to target a specific Torch version, or approximately two years of subsequent versions. For example, a kernel targeting Torch 2.9 Stable ABI supports Torch >= 2.9. Apache TVM FFI is the first framework supported outside of Torch. TVM FFI is a standardized ABI for kernels that interoperates with other frameworks such as PyTorch, Jax, and CuPy. This allows kernel developers to create kernels that work across frameworks.
Fundamentals of agent kernel development
Kernel builders and kernels complement the rise of agent kernel development, which leverages agents to create (optimized) kernels from scratch. Together, they support workflows that allow agents to perform kernel scaffolding, construction, benchmarking, and iterative optimization.
Agent kernel development is still in its early stages, and a suitable development loop will continue to evolve. That’s why simple, clear basics are especially important when you need to be able to easily incorporate your tools into the agent workflows and frameworks people use.
kernel-builder helps enforce structure on how to scaffold and use kernel source code to perform reproducible builds. This allows agents to work within predictable project layouts and repeatable workflows. Its CLI is also intended to be optimized for agents. For example, this might mean non-interactive commands and output that are easier for the agent to interpret programmatically. To this end, we also have backend-specific skills that allow our agents to deal with different backend idiosyncrasies. These skills help you understand backend-specific toolchains, compilation paths, and performance considerations.
Successfully building a kernel is not your only goal; you must ensure that you achieve real speedup over the target hardware baseline. Therefore, a successful build is only the first validation step. Typically, this target hardware can contain many different accelerators, even different families of the same accelerator.
This makes it important to evaluate results across hardware vendors and generations, if appropriate. Tight integration with HF Jobs makes this benchmarking process easy. Agents can use this integration to run benchmark suites, collect performance results, and compare them to defined baselines.
This way, the agent can run tests across different hardware configurations, get reliable feedback on the performance of the generated kernel, and identify what needs to be done. That feedback can inform the next optimization iteration.
Below are some examples of agent extension kernels. These illustrate the types of kernels that can be developed and evaluated through this workflow.
others
Preferences
Setting up the environment to build a kernel using kernel-builder can be daunting. To make it easier for users, we’ve added an installation script that allows you to set up your environment with one click. If you want to work with temporary instances, it’s worth following the Terraform setup guide.
system card for kernel
After the kernels are built, each kernel creates a system card that exposes useful information such as how the kernel is used and the exposed interfaces. When the kernel is pushed to the hub, this system card becomes the front part of the kernel.
Is the kernel compatible with my system?
It’s a question you ask again and again to plan things better. Use the has_kernel() method for this purpose.
from kernel import has_kernel
print(With kernel (“Kernel Community/Activation”version =1))
Returns a Boolean value. If you are looking for a more detailed explanation of why a particular kernel is not supported, use get_kernel_variants().
from kernel import get_kernel_variants, VariantAccepted
for decision in get_kernel_variants(“Kernel Community/Activation”version=1): name = decision.variant.variant_str
if is an instance(Decision, VariantAccepted):
print(debt”{name}: Compatible”)
Other than that:
print(debt”{name}: Rejected ({Decision.Reason})”)
It should print (depending on the machine you are using).
torch212-cxx11-cu130-aarch64-linux: Compatible torch210-cu128-x86_64-windows: Rejected (CPU (x86_64) does not match system CPU (aarch64)) torch211-cu128-x86_64-windows: Rejected (CPU (x86_64) does not match the system CPU (aarch64)) torch212-metal-aarch64-darwin: Rejected (OS (darwin) does not match the system OS (linux)) torch211-metal-aarch64-darwin: Rejected (OS (darwin) does not match the system OS (linux)) torch210-metal-aarch64-darwin: Rejected (OS (darwin) does not match system OS (linux)) torch29-metal-aarch64-darwin: Rejected (OS (darwin) does not match system OS (linux)) …
Improved manylinux_2_28 support
Kernel builders have targeted manylinux_2_28 almost from the beginning. Previously, I was targeting ManyLinux using the latest gcc toolchain compiled with glibc 2.28. I linked libstdc++ statically to avoid compatibility issues with older versions of libstdc++.
However, I’ve recently run into some issues with this approach. Some libstdc++ features use global initialization. This can lead to data corruption when multiple libstdc++ versions work, such as libstdc++ dynamically linked by PyTorch and libstdc++ statically linked by the kernel. Some modern kernels use features (such as C++ regular expressions) that trigger global initialization, resulting in data corruption, segmentation faults, and other issues.
To resolve this issue, the kernel now dynamically links libstdc++. The kernel is now compiled using the official manylinux_2_28 toolchain to ensure compatibility with older libstdc++ versions.
conclusion
The goal of our kernel project is to serve both kernel developers and users of custom kernels. We always welcome feedback from our community on how we can improve. Please feel free to contribute!
Acknowledgment: Thanks to Aritra for reviewing the post.
