SSH key forwarding using remote ssh with a dev container

[guylevin]

I use vscode to develop on a remote machine using ssh and running a dev container on that machine.

When using one of the pre-built dev containers, I can use the git key from my local machine to access github

When using my custom dev container, it does not work

ssh-add -l shows the correct key

But when connecting to github, it says:

ssh -o "ForwardAgent=yes" -vvvT git@github.com
...
debug3: ssh_get_authentication_socket_path: path ''/tmp/ssh-****/agent.202929''  
debug2: get_agent_identities: ssh_agent_bind_hostkey: communication with agent failed  
debug1: get_agent_identities: ssh_fetch_identitylist: communication with agent failed  
...

I've tried many things, so I'll write the base configuration:

devcontainer:

{
	"name": "Python 3",
	"build": {
		"dockerfile": "../Dockerfile"
}

dockerfile:

# Use an arm32v7 base image
FROM arm64v8/debian:bookworm

# Install necessary packages
RUN apt-get update && apt-get install -y \
    ...
    && rm -rf /var/lib/apt/lists/*

# Add a new user with UID and GID of 1000
RUN groupadd -g 1000 vscode && useradd -u 1000 -g 1000 -m -s /bin/zsh vscode

# Create the sudoers.d directory and add the sudoers file
RUN mkdir -p /etc/sudoers.d && echo "vscode ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/vscode && chmod 0440 /etc/sudoers.d/vscode

# Copy the requirements.txt file into the container
COPY requirements.txt /home/vscode/requirements.txt

# Create a virtual environment and install Python dependencies
RUN python3 -m venv /home/vscode/venv && \
    /home/vscode/venv/bin/pip install --no-cache-dir -r /home/vscode/requirements.txt

    
# Switch to the new user
USER vscode

# Install Oh My Zsh for the vscode user
RUN sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)" "" --unattended

# Set Zsh as the default shell
ENV SHELL=/bin/zsh

# Update the PATH environment variable
ENV PATH="/home/vscode/venv/bin:$PATH"
    
RUN eval "$(ssh-agent -s)"
...

ssh-add -l
3072 SHA256:****** ***@***.com (RSA)

When using the pre-built devcontainer image but adding steps to the docker file, the key also works

FROM mcr.microsoft.com/devcontainers/python:1-3.12-bullseye

# Install necessary packages
RUN apt-get update && apt-get install -y \
   ...
    && rm -rf /var/lib/apt/lists/*

...

What is missing from the custom image?