Bot should send typing while still generating completion #10

New issue

Closed

opened 2023-07-30 12:59:46 +02:00 by Wroclaw · 1 comment

Wroclaw commented

2023-07-30 12:59:46 +02:00

Owner

Currently, the bot sends a typing indicator when it starts to execute the completion request. If the competition request takes too long, visually in Discord it stops typing.
However, I think it should send a typing indicator when it's still generating a completion message.

A possible solution that I had in mind is to use token streaming, and send the typing if 5 seconds have passed.

Currently, the bot sends a typing indicator when it starts to execute the completion request. If the competition request takes too long, visually in Discord it stops typing. However, I think it should send a typing indicator when it's still generating a completion message. A possible solution that I had in mind is to use token streaming, and send the typing if 5 seconds have passed.