Steganosaurus - Linguistic steganography with LLM

Anyone can see when something is encrypted. But what if it looks like a basic text message? That was my thought behind this project - hiding the message inside another message.

Steganography

Essentially, steganography is a method to hide a secret message inside public media. Some examples are:
LSB - Hiding secret data in the last (least significant) bits.
Acrostic text - Reading the first letter of each word.
File append - hiding raw message in the end of the image file.


While they all work, there is one specific unique downside - they are created to be deciphered later. Moreover - it's pretty easy to do. BUT what if there is a new, modern way to create a publicly accessible text, which will be impossible to decipher, unless you have some sort of "key" while the cipher looks normal - without any indication of the existence of the secret message. That's when I decided to use LLMs to generate the readable ciphertext.

LLMs and the potential they hold

It is essential to understand the basics of AI to understand the next steps. But I'll try to make it easy: Whenever you generate any text using any large language model - your system gets a vector with "token" and "probability". It means that the top options are the most "suitable" or, more important for us - most "human-like", meaning - ordinary.

We don't want the output to look like it was written by human. What matters the most is avoiding the gibberish look. No one suspects AI written text of hiding something ;)

To make it even easier to understand, we could say that these vectors are what makes each model answer differently (which is a BIG simplification, but it works for our case). We could represent the example as:

Current text: The weather is
Options:
good | 0.6854
great | 0.2192
bad | 0.0138
...

Again, this is a big simplification. But the goal of every model is to create a useful and suitable answer based on the context. In this case, each token could work! Which is great for us, because we can go to the next step.

The workflow

We have to transform the message into a usable string. One good method is to use a binary sequence. Although we could use any other number-based code, we use binary to make it more suitable for different "smaller" LLMs, as picking the 3rd or even 16th option results in distinctly broken text. When the binary sequence is complete, the next step is to generate the message itself. Then, we need the prompt itself - something for the LLM to talk about. Next:
1. We give this prompt to the LLM and let it generate the first probability vector.
2. The LLM chooses either the 0. or 1. option, based on the next bit of our plaintext.
3. Repeat with every single bit of the string.

As a result, we get generated text that can be easily deciphered if: Your interlocutor uses the same LLM (even the version matters!) They use the same prompt They get the EXACT SAME TEXT - including symbols like \n, EOS, etc - they are part of the ciphertext too!

Advantages and disadvantages

There are several advantages of this method:
1. The cipher is realistically impossible to notice with ANY tool.
2. The prompt acts as the first key needed for decryption.
3. The LLM model acts as the second key needed for decryption.
4. Almost impossible to brute-force even in this basic configuration.

But there are also some downsides:
1. Slow generation.
2. Huge output text even for a small message.
Both are theoretically solvable if we compress the binary code.

Using another numeric sequence could work, but in most cases, it would make the ciphertext look unnatural, as the most realistic options in the vector are usually in the 0. or 1. position.

3. Hidden symbols (\n, EOS) count as tokens too. (Solvable, if "bad symbols" are ignored.)
4. When the message ends, the ciphertext cuts off too abruptly. So the ending looks unnatural.
5. The only way to decipher the text is leaking the exact same LLM model and prompt.
6. The prompt can be guessed based on context (solvable, if we add an additional "key" to encrypt and decrypt the generated text. ChaCha20 + Argon2id implementation is on my GitHub.)

Conclusion

The security of this scheme reduces to a simple constraint: the decoder must reproduce, bit-for-bit, the same probability distribution the encoder saw at every generation step. That means matching model weights, tokenizer, precision/quantization (a Q4 .gguf can rank candidates differently than Q8 or fp16 for the same prompt), sampling code path, and the full token history up to that point - since next-token probabilities are conditioned on everything before them.

In information-theoretic terms, each step contributes at most H(token) bits of channel capacity, where H is the entropy of the model's output distribution at that step. The current top-2 scheme extracts exactly 1 bit per token regardless of the real entropy available - which is why cover length scales linearly with message length rather than with the information content of the message. An arithmetic-coding approach (mapping the message to an interval within the full cumulative distribution, rather than a binary split of the top two candidates) would approach the theoretical channel capacity per token instead of a fixed 1 bit, at the cost of being more brittle to any mismatch between encoder and decoder distributions.

The prompt-as-key design has a concrete weakness: natural-language prompts have low entropy per character (roughly 1-2 bits/char for English, versus 128+ bits of true randomness in a cryptographic key), which puts short passphrases well within reach of offline dictionary attacks - especially with a fixed, project-wide salt. Deriving the encryption key via Argon2id (memory-hard, tunable cost) rather than using the prompt directly closes this gap, but a fixed salt still means every message sent under one passphrase falls together the moment that passphrase is recovered - there's no per-message forward secrecy without a unique nonce/salt per message.

Error propagation is a direct consequence of the autoregressive conditioning: because p(token_i | token_1...token_{i-1}) depends on the full prefix, any alteration to the cover text - a re-encoded whitespace character, a re-tokenized substring, even a byte-level change invisible to a human reader - desynchronizes the decoder's distribution from the encoder's at that point and beyond, with no error-correction layer to recover. This is why the cover text must be transmitted as an untouched file rather than retyped, copy-pasted through a lossy channel, or passed through anything that normalizes whitespace or Unicode.

None of this addresses traffic analysis: message length, timing, frequency, and the fact that two parties are exchanging LLM-generated text at all remain observable regardless of how well the payload is hidden inside it.