SillyTavern & Gemini API: Understanding Tavern Prompts
Our tavern SillyTavern is running from the command line, so we can see the underlying operations through the command line logs.
For example, when I sent the sentence "Set off! Head to the Soul-Breaking Mountain Range! The final trial to obtain the Moon Shadow Grass."
The command line log looks like this.
After this "Streaming request finished" is displayed, the tavern page starts showing the AI's reply content.
So in the command line log, the content between two "Streaming request finished" entries is what the tavern sent to the AI.
* Note: When viewing logs in screen, you cannot use pageup/pagedown directly to scroll up and down.
(Of course, you can use the mouse to drag and scroll, but if you just resumed with screen -r, the xshell window will only show one screen of information, and mouse dragging won't work either)
To view logs in screen in Linux, you need to use Ctrl + A, [ (left square bracket)
Saving logs from screen in Linux also requires a special method
Copy it out and take a look, it's a JSON structure. The large sections of content take up space, so we put it in a JSON formatting tool
https://tool.chinaz.com/tools/jsoneditor.aspx (This tool is used specifically because it automatically adds "" to keys)
* Note, you need to delete the "Streaming request finished" shown in the image below, then delete the "Google AI Studio request:". Only then is it valid JSON format
Let's take a look at what the tavern SillyTavern sends to the Gemini API
First, we can see the data is divided into 4 major sections.
Let's open them one by one and take a look.
The contents section is the conversations that have occurred.
You can see it distinguishes which parts were "said" by the AI and which were "said" by the user.
The last set of data in the contents section is the last sentence "said" by the user.
The next section is safetySettings
It looks like some toggle configuration items. Let me use a tool to translate it. You can see it disables restrictions on dangerous content, pornographic content, and such.
The following section is generationConfig
The last section is systemInstruction
As you can see, this is the character description part of the character card.
In summary, this means that every time the user "says" something (even just one sentence or a few words), the tavern constructs such a large section to send to the AI, including
A fairly large section of historical conversation records going back from what the user just "said",
Character description,
Requesting the AI to lift some restrictions such as pornographic content,
And some other parameters.
And the AI, under such a large prompt, generates a small section of reply. That is the latest content at the very bottom.
Of course, my explanation above is only a very superficial one. The tavern has many complex and refined features that I haven't used and don't understand.
But these analyses are enough for us to know that the essence of the tavern is just this. The AI side does not save memories specific to each user; the user needs to bring enough information as a prompt every time to make the AI generate new text that appears to conform to "memory".
The prompt context length is limited, so in principle, as the length of novels written with AI gets longer and longer, the AI will forget content from earlier times, and newly generated text may be completely unrelated to earlier content. Even with the same nouns/characters, there may be contradictions between earlier and later parts.
If you want to improve the quality of generation, you will inevitably need to use longer prompts, which brings higher usage costs and generation time.
========
Posts
