betterborg · the advanced_get plugin

Telegram Remote Shell

Send .a and a shell command to the bot. It runs the command with zsh on the machine the bot runs on and replies with the output. Files travel both ways: files you send become the command's input, and files the command writes come back to the chat. Only the bot's admins can use it.

Quick start

Open a private chat with the bot. (On a userbot, you can also type these in any chat from the userbot's own account.) Each line below is one message:

.a uname -a
.a df -h
.a seq 1 5000

The first two reply with the command's output. The third prints more than 4000 characters, so its output arrives as a .txt file.

To work on a file, send it with a caption that starts with .a, or reply with a .a message to a message that has the file:

.a wc -l *
.a gzip -k *

The bot saves the file into an empty directory made for this command and runs the command there, so * picks the file up. wc -l replies with a line count. gzip -k writes a .gz next to the original, and the bot sends that .gz back. The original is not sent back, because the command did not change it.

To fetch a file from the machine, copy it into the current directory:

.a cp /path/to/report.pdf .

Who can use it

Only the bot's admins. Messages from anyone else are ignored, with no reply.

On this page, the bot is the betterborg instance that runs this plugin. It can log in as a Telegram bot account, or as an ordinary user account; an instance logged in as a user is called a userbot. Its admins have nothing to do with Telegram's group admins: they are set in the bot's own configuration.

A .a message runs when both of these hold:

  • It is not a forwarded message. A forwarded command never runs, whoever forwarded it.
  • At least one of these is true:
    • the sender is listed as an admin in the bot's configuration, by user id or by username;
    • the message comes from the account the bot runs as. On a userbot, that means everything you send from that account, in any chat;
    • the message was sent in a group or channel that the configuration trusts as an admin chat. In such a chat, every member counts as an admin.

The restart commands (.x, .sbb and .xf) are stricter: only listed admins and the bot's own account may use them, and admin chats do not count. Guest mode is stricter again: it accepts only admins listed by numeric user id.

Commands

A command message starts with .a, then optional flag letters, then at least one space or line break, then the shell command. Everything after that, over as many lines as you like, is the command. A one-line message:

.a ls -la

And a message that spans several lines:

.a
for f in *.txt; do
  echo "$f has $(wc -l < "$f") lines"
done

A file's caption works the same way as a text message.

.a CMD
Run CMD in the shell pool, in its own scratch directory. This is the plain form; each flag below changes one thing about it.
.aa CMD
Run CMD in a new zsh -c process instead of the shell pool. Such a shell reads only the startup files zsh reads for non-interactive shells (like .zshenv), so functions and aliases set up for interactive use may be missing. Useful when every pool shell is busy or the pool is stuck.
.af CMD
Run CMD without a fork, in the first shell of the pool itself. Variables, functions, aliases and options it sets stay for the next .af command. The directory does not: each command still starts in its own scratch directory. .af commands run one at a time, so a stuck one holds up every later one.
.ad CMD
Turn off album mode: send each file the command leaves behind as its own reply, and honour the file name prefixes.
.an CMD

Put noglob in front of CMD, so zsh does not expand *, ? or […] in its first command. Handy for URLs:

.an curl -sL https://example.com/api?q=1
.k
Stop a running command. See Long commands and restarts. From a guest chat, @BOT .k (see Guest mode).
.x or .sbb
Restart the shell pool. See Long commands and restarts.
.xf
The same restart, with a different reply.
/settings
On a bot account, in your private chat with the bot: show your settings for live output and the renderer, with buttons to change them. In a group it only says where the panel is. A userbot has no /settings: it uses the defaults, or the values you set through a bot that runs on the same machine as the same operating-system user, since both read the same settings files.
/help
On a bot account: a short list of these commands.

Combining flags

Flags combine, but only in the order a, f, d, n. So .afd, .adn and .afdn work, while .adf or .ana is not recognised and the message is ignored. The f flag only matters in the shell pool, so .aaf behaves like .aa.

Details

  • Letter case does not matter for .a and its flags: .A ls works. The restart commands must be lowercase and alone in their message.
  • Smart punctuation is undone before the command runs: curly quotes (“ ” ‘ ’) become straight quotes and an em dash (—) becomes --. Phone keyboards that "fix" punctuation therefore do not break commands. An en dash (–) is left as it is.
  • The message must start with .a. The only exception: lines of the form [In reply to …] or [Forwarded from …] at the very top are skipped.
  • The bot reacts to new messages only. Editing a command message does not run it again; send it as a new message.

Sending files in

Every command gets a new, empty directory to run in, called its scratch directory on this page. It is a randomly named directory under dls/ in the directory the bot was started from. After sending the results, the bot deletes it with everything in it.

Before the command runs, the bot downloads these files into the scratch directory:

  • the file attached to the command message itself (a file sent with a caption that starts with .a);
  • the file in the message the command replies to;
  • every file in the same album as either of those.

Each file is saved as <message id>_<original name>. A file without a name, such as a photo, gets a generated one. Since the names are hard to predict, refer to inputs with wildcards such as * or *.zip:

.a unzip -l *.zip

Any program installed on the machine can be used this way.

The plugin sets no size limit of its own. Telegram's own limits on downloads and uploads still apply.

Output and files back

Text output

  • The bot replies to the command with what it printed, as plain text (Markdown is not rendered). Errors (standard error) are mixed into the same text.
  • The output is shown as a terminal would show it: a progress bar that redraws its line with a carriage return shows only its last state, and colours and other terminal escape codes are removed. The settings can turn this off.
  • Leading and trailing whitespace is trimmed. If 4000 characters or more remain, the output comes as a .txt file instead, with the caption "This message is too long, so it has been sent as a text file."
  • If the command printed nothing, the reply is The process exited N., where N is the exit code. When there is output, the exit code is not shown. End the command with ; echo "exit $?" to see it.
  • A command that runs longer than 2 seconds shows its output live: see Live output.
  • If something fails inside the bot rather than in your command, the reply starts with "Julia encountered an exception. :(" followed by a Python traceback.

Live output

A command that is still running after 2 seconds gets a preview: a reply that shows a header such as ⏳ #3, a blank line, and the last part of the output, ending in a ▌ cursor. It changes as the command prints. A faster command shows no preview, and its reply is exactly as described above.

  • In a private chat with a bot, the preview is a Telegram draft by default (settings can change this): a live preview that grows about once a second. It has a Stop button when the bot runs Telethon 1.45 or later and your app is Telegram 10.3 or later; pressing Stop stops the command. A preview without a Stop button says · .k to stop in its header. In groups, on a userbot, or where Telegram refuses a draft, the preview is a normal message that the bot edits every 2 seconds (every 4 in groups), then every 5 (10 in groups) once the command has run for 30 seconds. On a bot account that message has a ⏹ Stop button, which only admins may press (anyone else is told so); the button goes away when the command ends. A userbot cannot show buttons, so its preview says · .k to stop.
  • The header says waiting for a free shell while every pool shell is busy, and ⏹ #3 stopping… once the command is being stopped.
  • When the command ends, the preview becomes its output by default: the same text as above, or, at 4000 characters or more, the last part that fits in one message under the line "✂️ The full output is in the file below.", followed by the .txt file. An edited preview changes silently, without a notification. A draft becomes a new message, which notifies.
  • A stopped command's output ends with a note: ⏹ Stopped (exit 130)., or ⏹ Stopped: the bot is going offline (exit N). when the bot is shutting down or restarting. A command stopped while it waited for a free shell never runs and says ⏹ Stopped before it ran. (on a shutdown, ⏹ Stopped before it ran: the bot is going offline.)
  • While a bot's draft is live, Telegram for Android disables the send button, so a long command under drafts keeps you from typing until it ends or you press Stop.
  • Telegram apps keep one draft per chat for each bot, so two commands previewed as drafts at once in the same private chat overwrite each other's preview until one ends. Their outputs are not affected. Edits avoids this.
  • With the New reply setting, the output arrives instead as a new reply, exactly as for a fast command, and the preview is removed. If the bot cannot delete it (a bot cannot delete its own messages in a group after 48 hours), the preview is edited to "Finished; output below."
  • Programs that buffer their output when it is not a terminal show it in blocks, or only at the end. Ask them to flush each line: python3 -u, stdbuf -oL CMD, grep --line-buffered.
  • Commands run in guest mode show their output live in their answer too.
  • The bot's operator can turn live output off with borg_shell_streaming=0 (see For developers). On an old Brish library without streaming, .a and .af show no preview; .aa still does.

Settings

On a bot account, send /settings in your private chat with the bot. It shows each setting with what it does and costs, and buttons to change it; a press says the new value in a short pop-up and updates the panel. The settings are per admin and kept across restarts:

Live output in private chats: Drafts (the default) or Edits
Drafts are Telegram's live preview. While a bot's draft is live, Telegram for Android disables the send button, so a long command under Drafts keeps you from typing until it ends: press its Stop button, or choose Edits. Where the bot cannot give a draft a Stop button (it runs a Telethon older than 1.45), you cannot even send .k, so choose Edits if that matters.
Live output in groups: Edits (the default) or Drafts
Telegram allows drafts only in private chats today, so Drafts here falls back to edits.
When a command with a preview ends: Edit the preview (the default) or New reply
Edit the preview turns the preview into the output. An edited message changes silently, with no notification; a draft becomes a new message, which notifies. New reply sends the output as a new reply, which notifies, and removes the preview.
Renderer: On (the default) or Off
On shows output as a terminal would (progress bars show their last state, colours are removed). Off shows the output as the command wrote it, except that .aa turns each carriage return into a line break. This setting also applies to your guest mode answers; the others do not.

Each setting can also be typed: /settings private drafts (or edits), /settings groups edits, /settings final edit (or reply), /settings render off (or on).

Files

After the text, the bot sends every file left at the top level of the scratch directory, including files whose names start with a dot. Directories are skipped with everything inside them, so pack a directory with zip or tar to get it back.

Downloaded inputs that the command left untouched are deleted first, so they are not echoed back. "Untouched" means the file's modification time and size did not change and it was not replaced by a new file. To get an input back as it is, touch it or copy it under another name.

Album mode is how several files are sent by default: the bot groups them by extension and sends each group as one album (GIFs go one by one). These albums are not replies to your command.

When there is exactly one file, or with the .ad flag (album mode off), each file goes as its own message, in name order, as a reply to your command. In this mode a prefix in the file name chooses how Telegram shows the file:

voicenote-
a voice message
videonote-
a round video message
fdoc-
a plain file (a document). A photo sent this way is not compressed.
streaming-
a video marked as streamable, so it can start playing before it has downloaded

For example, reply to an audio file with this to get it back as a voice message (if ffmpeg is installed on the machine):

.a ffmpeg -i * -c:a libopus voicenote-reply.ogg

The input is untouched, so it is deleted, and the one file left goes out as a voice message.

Caveats

  • Everyone in the chat sees the results. In a group, every member sees the command, its output and its files.
  • Commands have full permissions. They run with all the permissions of the operating-system user the bot runs as. A mistyped rm does the same damage here as in a terminal.
  • Bots in groups. This is Telegram's rule, not the plugin's: in its default privacy mode, a bot account in a group sees only some messages, such as /commands and replies to its own messages. For .a messages to reach it, turn privacy mode off or make the bot a group admin.
  • Smart punctuation is replaced inside quotes too. .a echo "a—b" prints a--b.
  • The scratch directory is temporary. Write anything you want to keep to an absolute path outside it.
  • Exit codes are hidden whenever there is output.
  • No input, no time limit. See How commands run and Long commands and restarts.
  • Forwarded and edited messages never run.
  • Binary output. Bytes that are not valid UTF-8 show as \xNN escapes in the reply, and as "�" in a live preview. With live output turned off, raw binary data under .aa makes the bot report an exception instead. Write binary data to a file.

How commands run

Commands run with zsh, on the machine where the bot runs, as the operating-system user the bot runs as.

Shell pool. The bot keeps a set of long-lived zsh processes ready, managed by the Brish library. There are 16 of them unless the bot's operator sets another number. These shells started once, so whatever their startup files loaded (functions, aliases, environment variables) is already in place for every command.

Fork. For a plain .a, a free shell in the pool forks: it makes a throwaway copy of itself, and the command runs in the copy. A cd, a variable or a function that the command sets is gone when it ends, and the next command starts clean. .af skips the fork, and .aa skips the pool altogether (see Commands).

When every shell in the pool is busy, a new command waits until one is free.

Before each command, the shell changes into the scratch directory and sets the variable $jd to its path. If the shell has a function named jinit, it calls it there. Every pool shell has the environment variable JBRISH=y, so startup files can tell that they are serving the bot.

There is no way to send input to a running command, so programs that wait for typing (editors, pagers, password prompts) do not work. Under .aa the command's input is empty, so such a program reaches the end of its input at once and usually exits instead of waiting.

Long commands and restarts

  • The plugin sets no time limit. A command that never exits keeps its shell busy until you stop it, and its preview keeps showing its latest output.
  • .k stops a running command. Each command gets a number, shown in its preview's header (⏳ #3):
    • as a reply to the command, or to its preview, .k stops that command;
    • alone, it stops the only command running in this chat, or lists them when there are several;
    • .k 3 stops #3, .k all stops every command running in this chat, and .k ls lists them with their age and the start of the command.

    It sees the commands of this chat, whichever admin started them; in your private chat with the bot, also the commands you started in guest mode. The bot replies ⏹ Stopping #3…; the preview's header then says ⏹ #3 stopping…, and the output ends with ⏹ Stopped (exit N). A command that ignores the interrupt is sent stronger signals, so stopping it can take a few seconds. A command stopped while it waited for a free shell never runs. Only admins can use .k, as with .a.

  • On a bot account, a preview's Stop button does the same (see Live output); a pop-up confirms the press.
  • .k sees only commands that show live output: with live output turned off it stops nothing, and on an old Brish library it cannot stop .a or .af (it says so). Then find the process id and stop it from a second command. .aa works even when every pool shell is busy:
    .aa pgrep -l ffmpeg
    .aa kill 12345
  • When the bot shuts down or restarts (its server is stopped, or an admin sends .restart or .shutdown, alone in the message, in any letter case), it first stops every running command, waits up to 15 seconds for their output, and only then disconnects. The output of each ends with ⏹ Stopped: the bot is going offline (exit N). With live output on, a command sent while it waits never runs, and says ⏹ Stopped before it ran: the bot is going offline. A command that takes longer to stop is killed without a final message, as is every command when a bot started directly with python3 stdborg.py is stopped with Ctrl-C.
  • A stop reaches the command and everything it started, unless a program detached itself into a session of its own (a daemon, or setsid): such a program keeps running after the command is stopped, and after the bot restarts.
  • .x or .sbb starts a fresh shell pool for the commands that follow, and replies "Restarted brishes." .xf does the same and replies "Reinitialized brishes. Note that old running instances can still rejoin." Both also retire the other plugins' pool, which starts afresh at its next use.
  • The new shells read their startup files afresh. Restart after changing those files, or after an .af command left the shared shell in a bad state.
  • A restart does not cancel commands that are already running. With the current Brish library, the old pool shuts down in the background once those commands finish. The reply says how many commands still run on an old pool, if any, including pools that an earlier restart retired (2 commands still run on old pools; .k stops them.). Commands running in other chats are counted on a line of their own, since .k stops them only in their own chat, and your guest commands on another, since @BOT .k in their chat stops them. Commands that only another admin can reach (their guest commands, or commands in their private chat with the bot) are counted last, as commands that they can stop.

Guest mode: running from any chat

Guest mode is a Telegram feature that lets a bot answer where it is mentioned, even in a chat it is not a member of. With it, an admin can run a command from any chat by starting a message with the bot's username (written BOT below):

@BOT .a uptime
  • It works only when the bot is a bot account (not a userbot) and its operator has turned guest mode on for it.
  • The message must start with the mention, then at least one space, then .a with any flags (or .k, below). @BOT: .a … and @BOT.a … get a usage hint instead of running. A mention inside a code block or inline code does not count.
  • Only admins listed by numeric user id may use it. Anyone else who mentions the bot gets "Not available here."
  • A forwarded message, or one sent through another bot or a connected Business bot, never runs.
  • A command is dropped, never run, if the bot receives it more than 60 seconds after it was sent (for example because the bot was offline).
  • The bot first answers "⏳ Running…" in the chat, then edits that answer into the result. If it cannot post that first answer, the command does not run at all.
  • A command that runs longer than 2 seconds shows its output live in that answer, edited at the pace of an edited preview in that kind of chat (see Live output): every 2 seconds in a private chat and every 4 in a group, then every 5 or 10 once the command has run for 30 seconds. Its header says ⏳ #3 · @BOT .k to stop. The answer has no Stop button, and is never a draft. A stopped command's answer ends with ⏹ Stopped under the exit code. The renderer setting applies; the other settings do not.
  • Input files come from your message and from the message it replies to. Telegram sends only that one message of an album, so the other files of an album are not fetched, and the answer says so.
  • The answer is the output as plain text, cut to fit one Telegram message (4096 UTF-16 code units, which are roughly characters, with most emoji counting as two). A cut ends in "…". Lines under the output report a non-zero exit code (exit N), how many files were sent to your private chat, and whether the output was cut.
  • When the output is longer than about 3800 code units (the answer keeps room for the lines under it), the bot saves the whole output as output.txt in the scratch directory (under another name if the command made its own output.txt) and sends it with the other files.
  • Files go to your private chat with the bot, never to the chat where you called it, after a message that names the command. You need to have started the bot in a private chat first; if you have not, the answer says so. When there is exactly one file and the output is short enough for a caption (1024 UTF-16 code units), the file is also attached to the answer itself.
  • @BOT .k in the same chat stops a command you started there, with the forms of .k: alone it stops your only running command there (or lists them), and @BOT .k 3, @BOT .k all and @BOT .k ls work too. It sees only your own guest commands of that chat, and the bot answers with a short note there. .k in your private chat with the bot sees them as well.
  • A reply to the bot's answer does nothing unless it mentions the bot again.
  • On a userbot, betterborg replaces the "@" in outgoing text that looks like a guest shell call with a full-width "@", so text the userbot sends (command output, for instance) cannot run a command by accident. Text you type in a Telegram app is not affected.

For developers

Where the code lives

stdplugins/advanced_get.py holds the .a handler, the restart commands and the guest shell.

uniborg/util.py holds the shared pieces: running commands (brishz_capture, simple_run_capture), downloading inputs (run_and_get), sending output (send_output, discreet_send) and files (upload_output_files, send_files), the shell pool (init_brishes) and the admin checks (isAdmin, admin_cmd, is_admin_by_id). The built-in admin list is admins in that file.

Live output is built from uniborg/shell_stream.py (jobs and their output), uniborg/stream_driver.py (the paced preview, drafts and their Stop button) and uniborg/shell_settings.py (each admin's settings and the borg_shell_streaming switch). How they fit together is in docs/shell_streaming.md.

uniborg/guest_util.py and uniborg/tg_raw.py implement guest mode. Its design and safety rules are in docs/guest_mode.md.

Requirements
Python with the packages in requirements.txt (they include brish), and zsh on the PATH.
Running an instance
git clone https://github.com/NightMachinery/betterborg.git
cd betterborg
pip3 install -r requirements.txt

python3 stdborg.py         # standalone
python3 start_server.py    # or with the FastAPI server

stdplugins is the default plugin directory, so both load the shell with no extra settings. start_server.py serves stdborg:app with uvicorn on 127.0.0.1; uvicorn stdborg:app works too.

On the first start, Telethon asks on the terminal for a phone number (to log in as a user) or a bot token, and saves the login in a session file named after borg_session. Saving a .py file in the plugin directory reloads that plugin without a restart.

Environment variables
borg_plugin_path
The plugin directory to load. Default stdplugins, which contains this plugin.
borg_session
The session name, and so the session file. Default stdborg.
borg_brish_count
How many shells the pool has. Default 16.
borg_plugin_brish_count
How many shells the other plugins' pool has (.tex, ..ptv, the ebook processor and the like). Default 4. It starts on its first use, and .a never uses it.
borg_admins
Extra admins, comma-separated: numeric user ids, or usernames without the "@". Guest mode counts only the ids.
borg_log_chat
The numeric id of the chat that gets the bot's log messages. A bot account should set it. Without a usable log chat, a user account logs to its Saved Messages, and a bot account tries its admins' private chats.
borgp
The port of a SOCKS5 proxy on 127.0.0.1 to connect through.
borg_api_id, borg_api_hash
Telegram API credentials (TELEGRAM_API_ID and TELEGRAM_API_HASH also work). Built-in defaults are used when unset.
borg_shell_streaming
0 turns live output off: every command replies only when it ends, with its raw output, as before. false, no and off do the same; 1, true, yes, on, an empty value or unset leaves it on, in any letter case; any other value stops the bot at startup.
borg_guest_trigger_guard
0 turns off the "@" rewriting described under Guest mode. It takes the same words as borg_shell_streaming.

For example, an instance with its own session, a smaller pool and one extra admin:

borg_session=session_shell \
  borg_brish_count=4 \
  borg_admins=YOUR_USER_ID \
  python3 stdborg.py