Everyone has been using Claude and Openai as coding assistants for the past few years, but seeing the developments coming out of open source projects like Open R1 makes it less appealing. Looking at the LiveCodebench ratings below, you can see that the 7B parameter variant outweighs Claude 3.7 Sonnet and GPT-4o. These models are the daily drivers of many engineers in applications such as cursors and VSCODE.
Eval is amazing and everything, but I want to get my hands dirty and feel committed! This blog post focuses on how to integrate these models into your IDE now. Set up a smaller Olympic Coder 7B out of the two Olympic Coder variants and use the quantized variant for optimal local inference. Here is the stack we use:
Olympic Corder 7b. LMStudio Community 4-bit GGUF version of LM Studio: a tool that simplifies the execution of AI models Visual Studio Code (VS Code) VS Code Extension for Local Models
It is important to say that I chose this stack purely for simplicity. It is recommended to experiment with larger models and/or different GGUF files. Or even alternative inference engines such as llama.cpp.
The LM Studio is like a control panel for an AI model. It integrates with a facehub to pull the model, helping you find the right GGUF file, exposes an API that other applications can use to interact with the model.
In short, you can download and run them without any complicated setups.
Visit the LM Studio website: Open a web browser and go to https://lmstudio.ai/download. Select the operating system. Click the download button on your computer (Windows, Mac, or Linux). Install LM Studio: Run the downloaded file and follow the instructions. It’s like installing other programs.
The required GGUF files are hosted in the hub. You can open the model from the LMStudio hub using the Use this model button.
This will link to the LMStudio application and open it on your machine. Just choose quantization. I went to Q4_K_M. This is because it works well on most devices. If you have more calculations, try one of the options in Q8_*.
If you want to skip the UI, you can also load the model using LMStudio via the command line.
lms get lmstudio-community/olympiccoder-7b-gguf lms load olympiccoder-7b Start lms server
This is the important part. VSCODE must be integrated with the models provided by LMSTUDIO.
In LM Studio, activate the server in the (Developer) tab. This will expose the endpoint at http://localhost:1234/v1.
Install the VS code extension and connect to the local server. I went to condition.dev and there are other options too. In vscode, you will be taken to the Extensions view (click the square icon in the left sidebar or press Ctrl+Shift+x/cmd+shift+x). Search for “Continue” and install the extension from “Continue Development”. Open the (Continue) tab in the condition.dev and select (Add new chat model) in the Model dropdown. This will open the JSON configuration file. You must specify the model name. That is, olympicoder-7b
Most of VSCODE’s core AI features can be used, such as this setup:
Code Completion: Once you start typing, AI will suggest how to complete the code. Generate Code: Ask to write a function or an entire block of code. For example, you can enter (in the comment or chat window, depending on the extension). Refactoring Code: Ask AI to clean your code or make it more efficient. Write test: Ask AI to write unit tests for your code.
The Olympic coder is not Claude. Optimized with Codeforces-Cots datasets based on competitive coding challenges. In other words, you should not expect it to be very friendly and descriptive. Instead, expect a competitive, uncompetitive coder ready to roll up your sleeves and deal with tough issues.
Confuse the Olympic coder with other models for a rounded coding experience. For example, if you are trying to squeeze milliseconds from a binary search, try OlympicCoder. If you are designing users facing APIs, go to Claude-3.7-Sonnet or Qwen-2.5-Coder.
Share your favorite generation in the comments below. Try another variant of the Hub Olympic Coder. Try quantization types based on your hardware. Try multiple models at LM Studio for a variety of coding vibes! Try other VS code extensions like Cline with model catalog https://lmstudio.ai/modelsAddress feature
