Prototyping with AI Models on GitHub

GitHub provides a platform for developers to prototype, experiment with, and integrate AI models into their projects. This process involves several key steps and tools that facilitate the development and deployment of AI-powered applications.

List of References (Click to expand)

• Prototyping with AI models
• GitHub Models
• Github Marketplace Models Catalog

Table of Content (Click to expand)

• Overview
• Demo
  • Set Up Your Environment
  • Find and Select an AI Model
  • Experiment in the Playground
  • Integrate with Visual Studio Code
  • Make API Requests
  • Process and Use Responses
  • Transition to Production

Overview

Component	Description
Finding AI Models	You can find various AI models on the GitHub Marketplace. Navigate to the Models section to explore available models and view their details.
Experimenting with AI Models	GitHub offers a playground where you can test different models by adjusting parameters and submitting prompts. You can experiment with AI models using the API provided by GitHub.
Saving and Sharing Experiments	You can save your playground experiments and share them with others. This is useful for collaboration and getting feedback on your prototypes.
Integration with Visual Studio Code	GitHub models can be integrated into Visual Studio Code, allowing you to experiment with AI models directly within your development environment.
Going to Production	Once you are ready to move from prototyping to production, you can switch to using a token from a paid Azure account. This provides more robust and scalable options for deploying your AI models.
Rate Limits	There are rate limits in place to ensure fair usage of the AI models. These limits vary depending on the model and usage scenario. Monitoring your usage and optimizing requests can help you stay within these limits. The rate limits for the playground and free API usage are intended to help you experiment with models and prototype your AI application. For use beyond those limits, and to bring your application to scale, you must provision resources from an Azure account, and authenticate from there instead of your GitHub personal access token.

Demo

Set Up Your Environment

1. Create a GitHub Repository:

   • Go to GitHub and sign in to your account.

   • Click on the + icon in the top right corner and select New repository.

     

   • Name your repository and choose its visibility (public or private).

   • Click Create repository.

     

2. Generate a Personal Access Token (PAT):

   • Click on your profile picture in the top right corner and select Settings.

     

   • In the left sidebar, click Developer settings.

     

   • Click Personal access tokens and then Generate new token.

     

   • Select the scopes you need (e.g., repo, workflow, copilot) and click Generate token.

   • Copy the token and store it securely.

Find and Select an AI Model

1. Navigate to GitHub Marketplace:
2. Go to the GitHub Marketplace.
3. Click on the Models section to explore available AI models.

GitHub Marketplace	Model catalog

1. Select a Model:
2. Browse through the models and select one that fits your needs, such as OpenAI GPT-4.
3. Review the model’s details, capabilities, and usage instructions.

Browse models	Review model details

Experiment in the Playground

1. Access the Playground:
2. Go to the playground section for the selected model.

3. You can find this in the model’s details page on GitHub Marketplace.

   

4. Test the Model:

5. Adjust parameters such as prompt, temperature, and max tokens.
6. Submit prompts to see how the model responds.
7. Example prompt: Explain the basics of machine learning

• You can compare the performance between models:

First model	Comparison model

1. Save Experiments:
2. Save your experiments by clicking on the Save button.

3. Share the saved experiments with collaborators for feedback.

   

Integrate with Visual Studio Code

1. Install Extensions:
2. Open Visual Studio Code.
3. Go to the Extensions view by clicking on the Extensions icon in the Activity Bar.

4. Search for and install the GitHub Copilot extension.

   

5. Set Up API Key:

6. Open a terminal in Visual Studio Code.
7. Set your API key by running:

   export OPENAI_API_KEY="your-api-key"
   

Make API Requests

1. Write a Script:
2. Create a new file in your repository, e.g., ai_model_test.py.

3. Write a script to make API requests to the AI model. For example:

   import openai
   
   openai.api_key = "your-api-key"
   
   response = openai.Completion.create(
       model="gpt-4",
       prompt="Explain the basics of machine learning.",
       max_tokens=100
   )
   
   print(response.choices[0].text)
   

4. Run the Script: Run your script in the terminal to see the model’s response.

Process and Use Responses

Step	Description
Integrate Responses	- Use the responses from the AI model in your application. - For example, display the generated text in a web app or use it to automate a task.
Optimize and Iterate	- Continuously optimize your prompts and code based on the responses. - Iterate to improve the performance and relevance of the AI model.

Transition to Production

Step	Description
Provision Azure Resources	- Sign in to your Azure account. - Provision the necessary resources, such as Azure Cognitive Services.
Update Authentication	- Switch from using your GitHub PAT to an Azure production key. - Update your environment variable: export OPENAI_API_KEY="your-azure-api-key"
Monitor and Scale	Monitor your usage and scale your application as needed using Azure’s infrastructure