OK, this is what I ended up doing.
I wanted the remote server to have access to a local repository served over http. This works with any kind of traffic over tcp though, as it's not a web proxy but tcp forwarding.
I set up the files and served the website on the ssh client machine with PHPs build-in server. It's easy to use and requires no setup. You just start it in the base directory you want to serve. I used port 8000.
php -S localhost:8000
Then access the remove server from the client with the reverse tcp forwarding active.
Basically forwarding port 8000 on the remote host to port 8000 on the local ssh client.
ssh -R 8000:localhost:8000 remote_ip
But since I was connecting with a windows machine I used putty instead.
This is how you set up the tcp forwarding:
It seems like you can not only forward one port, but many ports and in whatever direction you want at the same time.
To try that you have things working:
In my case I wanted apt package manager to use the forwarded port so I just changed it to use http://localhost:8000 to access the packages.