Instructions to use princeton-nlp/SWE-Llama-13b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use princeton-nlp/SWE-Llama-13b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="princeton-nlp/SWE-Llama-13b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("princeton-nlp/SWE-Llama-13b")
model = AutoModelForCausalLM.from_pretrained("princeton-nlp/SWE-Llama-13b", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use princeton-nlp/SWE-Llama-13b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "princeton-nlp/SWE-Llama-13b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "princeton-nlp/SWE-Llama-13b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/princeton-nlp/SWE-Llama-13b

SGLang

How to use princeton-nlp/SWE-Llama-13b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "princeton-nlp/SWE-Llama-13b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "princeton-nlp/SWE-Llama-13b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "princeton-nlp/SWE-Llama-13b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "princeton-nlp/SWE-Llama-13b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use princeton-nlp/SWE-Llama-13b with Docker Model Runner:
```
docker model run hf.co/princeton-nlp/SWE-Llama-13b
```

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

language: en

datasets:

37 popular Python code repositories
See princeton-nlp/SWE-bench train split
See the make_datasets documentation on SWE-bench's GitHub for details on formatting input.

SWE-Llama

SWE-Llama are variants of the CodeLlama model fine-tuned on software engineering tasks extracted from real-world GitHub issues and pull requests. They were introduced and evaluated on the SWE-bench benchmark in this paper.

Model Details

Architecture: Transformer, based on CodeLlama architecture
Parameters: 7 billion for SWE-Llama-7b, 13 billion for SWE-Llama-13b
Objective: Generating patches to resolve GitHub issues, conditioned on issue description and code context

Training Data

SWE-Llama was fine-tuned on 19,000 issues and pull requests collected from 37 popular Python code repositories on GitHub, disjoint from those used in SWE-bench.

Training Procedure

Fine-tuned only the attention matrices using LoRA method
Trained for 4 epochs with a batch size of 32
Selected best checkpoint based on validation perplexity

Evaluation Results

When evaluated on the SWE-bench benchmark:

SWE-Llama-7b achieved 3.0% issue resolution rate using oracle context retrieval
SWE-Llama-13b achieved 4.0% issue resolution rate using oracle context retrieval

BibTeX Entry

@misc{jimenez2023swebench,
      title={SWE-bench: Can Language Models Resolve Real-World GitHub Issues?}, 
      author={Carlos E. Jimenez and John Yang 
        and Alexander Wettig and Shunyu Yao 
        and Kexin Pei and Ofir Press and Karthik Narasimhan},
      year={2023},
      eprint={2310.06770},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Downloads last month: 24

Spaces using princeton-nlp/SWE-Llama-13b 2

Papers for princeton-nlp/SWE-Llama-13b

SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

Paper • 2310.06770 • Published Oct 10, 2023 • 13

Code Llama: Open Foundation Models for Code

Paper • 2308.12950 • Published Aug 24, 2023 • 29