Category: Proxy

Followup roundup

Written by Fergal Hainey

I am thinking of reviving this blog. I am hearing from all angles why I should be publishing: in case what I share is useful, regular writing is supposed to be good for me, and because maybe “personal branding” is a fairly good idea after all.

The first thing I want to do is a draw a line under the old topics from 2012 while I was still at University of Aberdeen. I actually enjoyed reading them again, the topics are so unrelated to what I do now that they aren’t cringe worthy.

Boombox

My JVC ’80s boombox has had a good run. It has developed a hum on the tape deck that I can’t fix with my usual routine of poking the contacts with a spoon to ground them and twiddling the knobs to get static and dirt out. I don’t want to have to throw it away next time I move flat, so writing this has spurred me to contact a repair shop before I finished this paragraph.

In the end I used the FM transmitter, tape deck connector, and blank tapes pretty evenly depending on the circumstances. The FM transmitter if I needed to use my phone at the same time, the tape connector for better sound quality, and tapes for relaxing.

I have noticed an increased number of battery-powered small Bluetooth speakers that actually sound good. Finding something portable that sounded good without being too expensive is what drove me to buying a vintage boombox in the first place. The UE BOOM is the one I’ve heard the most of and I’m impressed by it. Some others sounded OK and there have been others that I didn’t think sounded good enough, but I didn’t take note of what they were. They seem to be in the £100 plus range still.

Node.js filtering proxy

I was really happy with how fast it is to get up and running with Node.js. I realise that forwarding all of your browser’s traffic isn’t a high load situation, but is more than Flask’s debug server would take with default settings for example. I appreciate that Node.js’ standard pattern worked so well out of the box, it’s evented and has a fast VM. Python is my first language now and I love it for how it encourages team maintainability, but often needs extra consideration to reach production performance. Python’s async syntax and PyPy should bridge the evented and VM gaps respectively. The new language features in ES6 should make JavaScript more attractive. Exciting times! I like to think of ease of maintenance by a team and performance as different facets of working at scale.

The code I wrote probably doesn’t work with 4OD anymore. I don’t have a television now so I don’t see adverts for shows, so I haven’t used any of the catch up services in years. It was a fun exercise though and gave me something to talk about in a job interview.

SD to HDD copier

I never tried it. I don’t hear the complaint of full SD cards so much anymore. I think the cloud and even lower SD card costs have solved the problem. I’m still excited about what new products these education focused electronics will lead to. The Arduino is still a great microcontroller. The Raspberry Pi is a great embeddable full computer and the new models are really powerful and still cheap. The new BBC micro:bit is awesome for experimenting with the components in modern mobile devices. I was worried that the micro:bit would be stepping on the Raspberry Pi’s toes, but after getting one at EuroPython 2016 I was won over. The micro:bit definitely has a focus on mobile devices, which makes a lot of sense as most people’s first computer is a mobile.

Universal repair USB stick

I feel guilty that I didn’t document my steps properly. I still have the one USB stick, but I’ve not had to use it in years. Modern Macs and Windows computers have built-in repair partitions. Upgrading to the latest OS is a lot more common now the upgrades are free and can be downloaded, so there seems to be fewer people with messed up systems. I myself hardly ever use a laptop anymore and do most of my work on a tablet. When a tablet goes bad, you just wipe it. That sounds like the worst case and evokes memories of reinstalling Windows. The reality is all the apps, settings, and documents are backed up on the cloud and starting from scratch again isn’t a big deal. With app stores for most platforms and Microsoft and Apple both peddling cloud storage, desktop could be going the same way. I’ve been considering trying to Dockerise my whole dev environment so even trickier installs are easier to reproduce. I’ve not seen anyone using Docker that way yet so could be interesting.

Clipboard sharing with Dropbox

I got a lot of use out of my little Dropbox app. Eventually messaging apps got better at accepting different sorts of pasted content and I stopped using it. I read the help page linked from my Public folder and it appears that only Pro and Business users can use Public folder simple URLs now, so my app may no longer work. If I were to make it now I’d probably have to use Dropbox’ API to make tokenised URLs. The help page also says that as of 2016-10-03 HTML documents won’t be rendered in a browser, I guess they’ll change the content type. My app relied on that for sharing rich text content.

The future

I do hope to keep this blog somewhat up to date now. Everyone in the tech industry has moments when they feel they were the first one to get a particular combination of parts working together, so I will try to share those times at least. I work mostly with Python, Docker, and AWS now so things could be more focused in that direction. That said I want to start making and computing as a hobby again not just work, I think about it so much.

Proxy followup

Written by Fergal Hainey

When doing my work on the filtering proxy, I did think about whether it could be made to work with HTTPS as well. The only useful plain text part of a HTTPS connection is the domain, so that seems to be a limiting factor on what I could do, just block domains. Obviously the rest of the data going through the proxy would be encrypted, but as my computer was the sender and intended recipient, surely I could decrypt it. As it turns out, Chrome (not my go to browser) can be configured to output a file containing they keys it uses in HTTPS. I discovered while using wireshark on another project. Wireshark can use this file to let you inspect HTTPS streams. Maybe my proxy could be updated to use this file (and parallels in other browsers if they’re there) to allow filtering for HTTPS.

A HTTP proxy with filters

Written by Fergal Hainey

This idea was sparked by Channel 4’s online player 4OD. It makes you watch adverts like you were watching a television, some before, some after, and even some in the middle. It makes sense I guess, they still have to get paid. Except all this is happening on my computer, so I’m sure I can do something about it.

First thought, ad blockers. Don’t work, it’s all stuff within a flash object I essentially still want to use (to watch the actual content) and all the ad blockers I can find are actually still pretty simple, they just block URLs.

Some WebKit “Web Inspector”ing and a bit of wiresharking later and I find the URLs of the adverts being shown. They’re all from CDNs and have complicated session URLs, so I make a general pattern for their URLs to give to an ad blocker. When the video files are blocked, the 4OD player declares there is a problem (which I guess there is really), and stops playing. So it’s going to be more complicated than that.

More wiresharking. Let’s find out where the flash object is getting these video URLs from. From an XML document, with a nice clear structure showing where the adverts are and when to play them. If only there was a way of just removing that <adverts> tag. A browser extension (like greasemonkey or Safari extensions) could change the content of stuff fetched by the browser, but the XML document is being requested and processed by the flash plugin, so that’s out. Some kernel extension to intercept network calls? Sounds messy and beyond me. A proxy, except a misbehaving one that alters content before it forwards it (like those stories of those American ISPs that do the opposite of what I’m trying to do, inserting ads)? That could work. Is there something that already does this? Maybe, but uncertain if it’s configurable enough, and it costs. So that’s out. Can I make one? Good question.

I’d been hearing a lot about Node.js, a platform based around javascript and designed to make the evented model of javascript (like AJAX) available for use with general programming, especially IO. It was a language I already had a lot of experience with, and had libraries for HTTP requests and responses. And the evented model was supposed to make stuff like this fast and efficient. So should be perfect.

The way proxies work is pretty intuitive, and there’s plenty of documentation about it. Making a simple one in node took less than 50 lines, and it worked! Adding filtering rules was simple, I didn’t need to configure my proxy, I could just rewrite it however I wanted! The first thing I tried was removing the <adverts> tag in URLs that matched the 4OD XML info document and it worked. The adverts simply disappeared! A bit of smartening up and I had an easy to use (for me) system to handle any task like this when I need it!