Skip to content

[Improvement] Add support for o1-preview and o1-mini OpenAI models #725

Closed

Description

@dougcunha

OpenAI has offered API access to its o1-preview and o1-mini models for some time now. However, differences in API usage make the current implementation incompatible with these models, resulting in the error I've attached below.

Failed to generate commit message: System.Exception: AI service returns error code BadRequest
at SourceGit.Models.OpenAIService.Chat(String prompt, String question, CancellationToken cancellation) + 0x3f9
at SourceGit.Commands.GenerateCommitMessage.GenerateChangeSummary(Change change) + 0x90
at SourceGit.Commands.GenerateCommitMessage.Result() + 0x19a

Activity

  1. love-linger commented on Nov 20, 2024

    @love-linger
    Collaborator

    Seems like #630

  2. self-assigned this
    on Nov 20, 2024
  3. love-linger commented on Nov 20, 2024

    @love-linger
    Collaborator

    Can you test the OpenAI api from curl?

    curl https://api.openai.com/v1/chat/completions \
        -H "Content-Type: application/json" \
        -H "Authorization: Bearer YOUR_API_KEY" \
        -d '{
            "model": "MODEL",
            "messages": [
                {
                    "role": "system",
                    "content": "xxxx"
                },
                {
                    "role": "user",
                    "content": "xxxxx"
                }
            ]
        }'
    
  4. dougcunha commented on Nov 20, 2024

    @dougcunha
    ContributorAuthor

    I apologize for the confusion. I received an email from OpenAI stating that I have access to the o1-mini, but a cURL request returned that it is not available for me yet.

    You can delete this. Thank you.

    {
        "error": {
            "message": "The model `MODEL` does not exist or you do not have access to it.",
            "type": "invalid_request_error",
            "param": null,
            "code": "model_not_found"
        }
    }
    
  5. added
    invalidThis doesn't seem right
    and removed
    need-more-infoExtra information or replay is needed
    on Nov 20, 2024
  6. love-linger commented on Nov 20, 2024

    @love-linger
    Collaborator
  7. love-linger commented on Nov 20, 2024

    @love-linger
    Collaborator

    PS. I've pushed a commit that outputs the response body if the AI service returns an error code.

  8. dougcunha commented on Nov 20, 2024

    @dougcunha
    ContributorAuthor

    My apologies. I did not use the correct model. A different error has now occurred.

    {
      "error": {
        "message": "Unsupported value: 'messages[0].role' does not support 'system' with this model.",
        "type": "invalid_request_error",
        "param": "messages[0].role",
        "code": "unsupported_value"
      }
    }
    
  9. love-linger commented on Nov 20, 2024

    @love-linger
    Collaborator

    Since the official OpenAI library is available for .NET 9, I'll try to refactor the AI integration.

  10. dougcunha commented on Nov 20, 2024

    @dougcunha
    ContributorAuthor

    I found this:

    This error occurs because the o1 series models, including the o1-mini, do not support messages with the "system" role in the chat completions endpoint. Currently, these models only accept messages with "user" and "assistant" roles.

    Differences from GPT-4 usage:

    GPT-4 supports messages with the "system" role to define model behavior; o1 models do not currently offer this support.

    Adapting your code:

    To use the o1-mini model, remove messages with the "system" role from your request. If you need to provide specific instructions to the model, include them directly in the user message content.

    {
      "model": "o1-mini",
      "messages": [
        {
          "role": "user",
          "content": "Você é um assistente que fornece respostas concisas. Minha pergunta é: [sua pergunta aqui]"
        }
      ]
    }
    

    Therefore, you integrate the desired instructions directly into the user's message, circumventing the current limitation of o1 models regarding the "system" role.

    I agree with you, I think that using the official OpenAI library is the best approach.

  11. love-linger commented on Nov 20, 2024

    @love-linger
    Collaborator

    I've pushed a branch feat-openai. Can you test it for me?

    PS. I might not reply immediately because it's already 10pm in China...

  12. added
    bugSomething isn't working
    and removed
    invalidThis doesn't seem right
    on Nov 20, 2024
  13. dougcunha commented on Nov 20, 2024

    @dougcunha
    ContributorAuthor

    It's working just fine, but I notice a difference compared to the previous implementation.
    Look for yourself.

    SourceGit_IOyYZOMZaU

  14. love-linger commented on Nov 20, 2024

    @love-linger
    Collaborator

    Unfortunately, I found that the OpenAI official library was not ready to publish with native aot mode...

    I'll modify the previous implementation for this issue tomorrow

  15. dougcunha commented on Nov 20, 2024

    @dougcunha
    ContributorAuthor

    After playing a little bit with the current implementation using the OpenAI lib I got this error once.

    Failed to generate commit message: System.Net.Http.HttpIOException: The response ended prematurely. (ResponseEnded)
    at System.Net.Http.HttpConnection.FillAsync(Boolean async)
    at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.Read(Span1 buffer) at System.ClientModel.Internal.ReadTimeoutStream.Read(Byte[] buffer, Int32 offset, Int32 count) at System.IO.Stream.Read(Span1 buffer)
    at System.Net.ServerSentEvents.SseParser1.FillLineBuffer() at System.Net.ServerSentEvents.SseParser1.Enumerate()+MoveNext()
    at OpenAI.Chat.InternalStreamingChatCompletionUpdateCollection.StreamingChatUpdateEnumerator.MoveNext()
    at OpenAI.Chat.InternalStreamingChatCompletionUpdateCollection.GetValuesFromPage(ClientResult page)+MoveNext()
    at System.ClientModel.CollectionResult1.GetEnumerator()+MoveNext() at SourceGit.Models.OpenAIService.Chat(String prompt, String question, CancellationToken cancellation, Action1 onUpdate) in D:\projetos\sourcegitOfficial\src\Models\OpenAI.cs:line 112
    at SourceGit.Commands.GenerateCommitMessage.GenerateChangeSummary(Change change, StringBuilder summary, StringBuilder body) in D:\projetos\sourcegitOfficial\src\Commands\GenerateCommitMessage.cs:line 76
    at SourceGit.Commands.GenerateCommitMessage.Exec() in D:\projetos\sourcegitOfficial\src\Commands\GenerateCommitMessage.cs:line 50

  16. love-linger commented on Nov 21, 2024

    @love-linger
    Collaborator

    Done. You can download the latest CI build.

  17. love-linger commented on Nov 21, 2024

    @love-linger
    Collaborator

    I will follow up on the issue - openai/openai-dotnet#243

    Once it is resolved, I will use OpenAI's official .NET library to replace the existing implementation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions