Fake Curl Shell for Network-Restricted VPS DD Scripts

Preface

In the DD script group

A group member encountered the problem that the script easily gets interrupted when downloading resources due to a poor network.

Idea

In the previous article we found that we cannot simply put a certain resource in a specified location on the VPS in advance
because different resources may end up being placed in the same location on the VPS.

So let's change our approach.
Throughout the execution of the entire script, the content and order of all curl commands are fixed.
In other words, every time the script is executed, the content of the 1st, 2nd, 3rd, ... nth curl command is fixed.
Even if a certain curl statement in the script is placed in a loop and executed multiple times, from the perspective of the unrolled execution sequence, the content of each curl command is fixed.

Practice

So we
add such a curl shell at the beginning of the existing script:
1) Record and print a call sequence number. The sequence number increases by 1 each call.
2) Output the pwd current directory
3) Output the complete curl command and all its parameters
4) This curl shell does not actually download the file.
5) Based on the call sequence number, execute a preset command. For example, 
cp file1 /path/to/file
or
cat file2
to replace the file-saving commands of curl -LO or curl -Lo
or the stdout-outputting command of curl -L
6) These preset commands are manually edited and added. Use case to implement the logic in 5).
7) This curl shell always returns success. 

Send this requirement to GPT, and we get
# Global call counter
curl_call_count=0

curl() {
    curl_call_count=$((curl_call_count + 1))

    # Output the call sequence number and current directory (to stderr)
    echo "[DEBUG] curl call # $curl_call_count" >&2
    echo "[DEBUG] Current directory: $(pwd)" >&2

    # Output the complete command (with all parameters, safely escaped)
    printf '[DEBUG] Command: curl ' >&2
    printf '%q ' "$@" >&2
    printf '\n' >&2

    echo >&2

    # Execute preset command based on the call sequence number
    case "$curl_call_count" in
         1)
             # Command executed on the 1st call
             # cp 1 /path/to/file"
             ;;
         2)
             # Command executed on the 2nd call
             # cat 2
             ;;
        *)
            # No preset command set
            echo "[DEBUG] 未设置预设命令" >&2
            # Can call the real curl
            # command curl "$@"
            ;;
    esac

    # Do not perform the real download, only print the information
    # Always return success
    return 0
}
Modify the original resinstall.sh script to look like this

Upload the modified reinstall-edited-2.sh to the VPS and test it
bash reinstall-edited-2.sh debian 13
debian 13 is just an example.
And we got an error log like this.

Suppose this VPS cannot access www.qualcomm.cn for some reason.
We want to change it to access www.cloudflare.com
Then we change case 1 to this
command curl -L http://www.cloudflare.com/cdn-cgi/trace

Try bash reinstall-edited-2.sh debian 13 again
The log strangely prints 2 sequence number 1s

Asked GPT, the guessed reason is pipe commands like curl xxx | something
GPT suggested that we change it to a temp file.
# Global call counter
COUNT_FILE="/root/curl_call_count.tmp"
echo "0" > "$COUNT_FILE"

curl() {
    curl_call_count=$(cat "$COUNT_FILE")
    curl_call_count=$((curl_call_count + 1))
    echo "$curl_call_count" > "$COUNT_FILE"
Now the sequence numbers are correct.

Let's tackle them one by one.

Sequence number 2 is
[DEBUG] curl call # 2
[DEBUG] Current directory: /root
[DEBUG] Command: curl -Lo /reinstall-vmlinuz https://deb.debian.org/debian/dists/trixie/main/installer-amd64/current/images/netboot/debian-installer/amd64/linux
Download https://deb.debian.org/debian/dists/trixie/main/installer-amd64/current/images/netboot/debian-installer/amd64/linux using our method, then upload it to the VPS at /root/ with the filename 2
Then write the script's case 2) like this
cp /root/2 /reinstall-vmlinuz

By the same logic, handle sequence number 3. Download the file, upload it to /root/ with the filename 3
cp /root/3 /reinstall-initrd
For sequence numbers 4, 5, 6, 7, 8, 9, they request resources from GitHub. Suppose our VPS cannot access GitHub, so we need to use a GitHub proxy, for example: https://gh-proxy.com/
Then write the case like this
4|5|6|7|8|9)
    local args=()
    for arg in "$@"; do
        if [[ "$arg" =~ ^https?:// ]]; then
            args+=("https://gh-proxy.com/$arg")
        else
            args+=("$arg")
        fi
    done
    command curl "${args[@]}"
    ;;
For sequence number 10, the original command is curl -L, so download http://deb.debian.org/debian/dists/trixie/main/debian-installer/binary-amd64/Packages.gz using our method, then upload it to the VPS at /root/ with the filename 10
Then in the case, use cat
10)
    cat /root/10
    ;;
Now try running bash reinstall-edited-2.sh debian 13
A new sequence number 11 appears
Same handling as sequence number 2. Omitted.

Try running bash reinstall-edited-2.sh debian 13 again
A new sequence number 12 appears
Same handling as sequence number 2. Omitted.

Try running bash reinstall-edited-2.sh debian 13 again
A new sequence number 13 appears
Same handling as sequence number 10. Omitted.

Try running bash reinstall-edited-2.sh debian 13 again
A new sequence number 14 appears
Same handling as sequence number 2. Omitted.

Try running bash reinstall-edited-2.sh debian 13

All done!


Reboot the VPS
After waiting for a while, log in again. 
No problem, the DD succeeded!

========
update
The solution that intercepts URLs via PATH for wget has been uploaded to GitHub
The v1 solution does not work for reinstall.
When using the v2 solution with reinstall.sh, the PATH setting in reinstall.sh needs to be modified.
export PATH="/tmp/fakebin:$PATH"

========

Still want more

Comments