Fake Curl Shell for Network-Restricted VPS DD Scripts

Preface

In the DD Scripts group

A group member encountered the issue that the script's resource downloads tend to get interrupted when the network is bad.

Approach

In the previous post we found that we can't simply put a resource at a specified location on the VPS in advance

because different resources may be placed at the same location on the VPS.

So let's try a different approach.

Throughout the execution of the whole script, the content and order of all curl commands are fixed.

In other words, every time we run the script, the content of the 1st, 2nd, 3rd, ... nth curl command is fixed.

Even if a curl statement is placed inside a loop in the script and executed multiple times. From the perspective of the unrolled execution sequence, the content of each curl command is fixed.

Practice

So we

add a curl wrapper like this at the top of the existing script:
1) Record and print a call sequence number. Increment by 1 on each call.
2) Output the current directory from pwd
3) Output the complete curl command and all of its arguments
4) This curl wrapper does not actually download the file.

5) Based on the call sequence number, execute a preset command. For example,
cp file1 /path/to/file
or
cat file2
to replace curl -LO or curl -Lo file-saving commands
or curl -L stdout-output commands
6) These preset commands are added by manual editing. Use case to implement the logic in 5).

7) This curl wrapper always returns success.

Send this requirement to GPT, and we get

# Global call counter
curl_call_count=0


curl() {
    curl_call_count=$((curl_call_count + 1))


    # Output call sequence number and current directory (to stderr)
    echo "[DEBUG] curl call # $curl_call_count" >&2
    echo "[DEBUG] Current directory: $(pwd)" >&2


    # Output the complete command (including all arguments, safely escaped)
    printf '[DEBUG] Command: curl ' >&2
    printf '%q ' "$@" >&2
    printf '\n' >&2


    echo >&2


    # Execute preset command based on the call sequence number
    case "$curl_call_count" in
         1)
             # Command to execute on the 1st call
             # cp 1 /path/to/file"
             ;;
         2)
             # Command to execute on the 2nd call
             # cat 2
             ;;
        *)
            # No preset command set
            echo "[DEBUG] No preset command set" >&2
            # You can call the real curl
            # command curl "$@"
            ;;
    esac


    # Don't perform the real download, only print info
    # Always return success
    return 0
}

Modify the original resinstall.sh script to look like this

Upload the modified reinstall-edited-2.sh to the VPS, and test

bash reinstall-edited-2.sh debian 13

* debian 13 is just an example.

Got an error log like this.

Assume this VPS cannot reach www.qualcomm.cn for some reason.

Change it to access www.cloudflare.com instead.

So we change case 1 to this

command curl -L http://www.cloudflare.com/cdn-cgi/trace

Try again: bash reinstall-edited-2.sh debian 13

Strangely, the log printed sequence number 1 twice

Ask GPT, the guessed reason is piped commands like curl xxx | something

GPT suggests switching to a temporary file.

# Global call counter
COUNT_FILE="/root/curl_call_count.tmp"
echo "0" > "$COUNT_FILE"


curl() {
    curl_call_count=$(cat "$COUNT_FILE")
    curl_call_count=$((curl_call_count + 1))
    echo "$curl_call_count" > "$COUNT_FILE"

Now the sequence numbers are correct.

Let's tackle them one by one.

Sequence number 2 is

[DEBUG] curl call # 2
[DEBUG] Current directory: /root
[DEBUG] Command: curl -Lo /reinstall-vmlinuz https://deb.debian.org/debian/dists/trixie/main/installer-amd64/current/images/netboot/debian-installer/amd64/linux

Download https://deb.debian.org/debian/dists/trixie/main/installer-amd64/current/images/netboot/debian-installer/amd64/linux using our method, then upload it to the VPS at /root/ with the filename 2

Then write case 2) in the script like this

cp /root/2 /reinstall-vmlinuz

Same approach for sequence number 3. Download the file, upload to /root/ with the filename 3

cp /root/3 /reinstall-initrd

For sequence numbers 4 5 6 7 8 9, they are GitHub resource requests. Assume our VPS cannot reach GitHub and needs a GitHub proxy. For example: https://gh-proxy.com/

Then write the case like this

4|5|6|7|8|9)
    local args=()
    for arg in "$@"; do
        if [[ "$arg" =~ ^https?:// ]]; then
            args+=("https://gh-proxy.com/$arg")
        else
            args+=("$arg")
        fi
    done
    
    command curl "${args[@]}"
    ;;

For sequence number 10, the original command is curl -L, so we download http://deb.debian.org/debian/dists/trixie/main/debian-installer/binary-amd64/Packages.gz with our method, upload to /root/ on the VPS with the filename 10

Then use cat in the case

10)
    cat /root/10
    ;;

Try again: bash reinstall-edited-2.sh debian 13

A new sequence number 11 appeared

Same handling as sequence number 2. Omitted.

Try again: bash reinstall-edited-2.sh debian 13

A new sequence number 12 appeared

Same handling as sequence number 2. Omitted.

Try again: bash reinstall-edited-2.sh debian 13

A new sequence number 13 appeared

Same handling as sequence number 10. Omitted.

Try again: bash reinstall-edited-2.sh debian 13

A new sequence number 14 appeared

Same handling as sequence number 2. Omitted.

Try again: bash reinstall-edited-2.sh debian 13

Done!

Reboot the VPS

After waiting for a while, log back in.

No problem, DD succeeded!

========

update

The solution that intercepts url wget via PATH has been uploaded to Github

https://github.com/crazypeace/fake-curl-wget-sh

The v1 solution doesn't work for reinstall.

The v2 solution, when used with reinstall.sh, requires modifying reinstall.sh's PATH setting.

export PATH="/tmp/fakebin:$PATH"