Mistral7b response starts with an extra leading space when streamed with Ollama

题意:当使用Ollama进行流式传输时,Mistral7b的响应以额外的前导空格开始

问题背景:

When I stream the response of mistral7b LLM with Ollama, it has an extra space to the left on the very first streamed chunk. Below is my code:

import ollama

stream = ollama.chat(
  model='mistral',
  messages=[{'role': 'user', 'content': 'Name an engineer that passes the vibe check'}],
  stream=True
)

for chunk in stream:
  print(chunk['message']['content'], end='', flush=True)

The output looks like this:

$ python3 test.py 
 Elon Musk, the CEO of SpaceX and Tesla, is an engineer who seems to pass the "vibe check." He is known for his innovative ideas in renewable energy, space travel, and transportation. However, it's important to remember that personality and vibes can be subjective, so not everyone may agree with this assessment. Additionally, Musk's public image should not overshadow the contributions of countless other engineers who are equally impressive but less well-known.

Note that very first empty space before the letter "E". How do I remove it?

问题解决:

Use lstrip() to remove the white space.

import ollama

stream = ollama.chat(
  model='mistral',
  messages=[{'role': 'user', 'content': 'Name an engineer that passes the vibe check'}],
  stream=True
)
first = True
for chunk in stream:
    if first:
        chunk = chunk['message']['content'].lstrip()
        print(chunk)
        first = False
    else:
        chunk = chunk['message']['content']
        print(chunk)

最近更新

  1. docker php8.1+nginx base 镜像 dockerfile 配置

    2024-07-11 07:54:03       101 阅读
  2. Could not load dynamic library ‘cudart64_100.dll‘

    2024-07-11 07:54:03       108 阅读
  3. 在Django里面运行非项目文件

    2024-07-11 07:54:03       91 阅读
  4. Python语言-面向对象

    2024-07-11 07:54:03       98 阅读

热门阅读

  1. 深入理解基本数据结构:栈详解

    2024-07-11 07:54:03       24 阅读
  2. 树形结构的一种便捷实现方案

    2024-07-11 07:54:03       28 阅读
  3. Julia 日期和时间

    2024-07-11 07:54:03       22 阅读
  4. linux下解压命令

    2024-07-11 07:54:03       26 阅读