Sneakerbots, Chrome Extensions, and the Evented Model
I like Air Jordan 1s. A lot. It's a new thing, especially since I hadn't owned a pair of shoes that wasn't all black since 1998, probably, but I'm enjoying this opportunity to step outside of my comfort zone and try something different.
Did you know that "sneakerbots" are a thing? Cause they are a thing. Limited edition sneakers are announced ahead of time and like any limited, premium item, there is a market around their buy and sell, so, naturally, a market has emerged around the very means to buy them in the first place. I watched a few YouTube videos and from what I can tell, "sneakerbots" look like applications that watch the clock and then zoom through the checkout process as soon as the shoes go on sale. The kicker is that few of them are free. Buyers often pay $25-$75 for a license that claims to work with all of the big retailers' websites.
I've been reading /r/sneakers a lot lately and there seems to be some misunderstanding about how they work. In particular, it seems like a lot of people have this idea that sneakerbots are some magic tools that guarantee a successful purchase, but the truth is that they are limited by the web sites' servers' abilities to process requests. This, ultimately, is the greatest weakness of these things because it means that no matter how good your script is, you're always at the mercy of the server's ability to handle traffic. At the same time, it stands to reason that traffic will ramp up in the milliseconds after a pair goes on sale, so the faster you can hit submit, the better your chance of completing a purchase.
Regardless of whether paid sneakerbots are worth it, the whole thing bugs me. It's fucked up that most new releases are only accessible through eBay at 3x or more their original price. It's fucked up that I can buy new releases from literally hundreds of different auctions for $500+ but I've not once seen a pair on someone's feet in Manhattan or Brooklyn. These new releases are going straight from the shelves to eBay and I'm guessing that most will never see concrete. The paid sneakerbot market adds insult to injury, exploiting the desire to catch a drop without being able to guarantee a sale. The videos I saw looked shady as shit: you're installing some closed source software built anonymously and designed to exploit shopping carts. It has access to your browser, your account, probably your credit card info. Fuck everything about that.
Enter FSB, my open-source sneaker buyer in the form of a Chrome extension. I had been looking for an excuse to learn a bit more about extensions anyway and if I can give an alternative to paid shady software, that's cool. (I guess it'd be better if it was a bit easier to use, but whatever.) My current version is shitty: it targets Footlocker's mobile site, it only works with shoes, and there might be a bug that's keeping the checkout button from being clicked. (In fairness, it's a bug caused by Footlocker's mobile site, a mismatched cert name that stops JS execution.) I threw it together in a bit more than a day and when I tried to use it the morning of the Shattered Backboards release, Footlocker's shitty servers were a show-stopper. At the end of this post, I'll talk about how it could be improved and what it would take to build a better sneakerbot. You can find the Github repo at https://github.com/subvertallchris/fsb.
I'll walk through some of the things that took time to figure out.
Start by reviewing the docs on extension setup, I won't cover that since there are many good tutorials out there for that.
My manifest.json likely demands more permissions than it. It uses a non-persistent background page to maintain state, listen for events, and fire off new pieces of the process as events occur.
cartcheckout.js both act as registration scripts. They ping the background page so it can record their tab id and, if needed, start a process when it discovers them.
Chrome's extensions API makes it very easy to manipulate CSS and start scripts, but if you want to, say, get the value of a field from a form and use it somewhere else, or make the clicking of a button on tab 1 cause a button on tab 2 to be clicked, it can be tricky. background.js contains the bulk of the code that solved this problem and, mentioned above, feels a lot like how we use RabbitMQ to kick off actions on servers at work.
Evented Programming for Dummies
Most programmers, myself included, typically approach tasks from a procedural perspective. We think in terms of sequence: first you do A, then B, then C, and so on. In the Ruby language, we have something like this:
def a_method do_this then_this finally_this end def do_this # some things happen end def then_this # some more things happen end def finally_this # final things happen end
An evented model decouples one action from another. Instead of defining the sequence, you create independent actions that announce their behavior through messages, or events. Each action can listen for and respond to others. Think of it as, "When Section A completes, it announces, 'SECTION_A_DONE'." Anything that is listening for "SECTION_A_DONE" will start working on its own. Section A neither knows nor cares what happens after it and those actions that start don't care what fired the message, they just know that it's their turn to do something. We also do away with the idea of a central function that ensures that Section A flows into Section B. At my day job, we use a server called RabbitMQ to manage this process and some programming languages, like Go, have messaging baked in as core concepts.
With FSB, I had a few different tasks happening independently of one another: I needed a cart open, I needed something else watching a timer and prepared to POST a form, I needed to listen for a checkout and complete the purchase when the cart was populated. There are some procedural elements to it but approaching it from an evented perspective let me think in terms of state, not process. This worked well with Chrome's Extensions API. Tabs are able to send messages to the persistent background page, and the background page can send messages back to tabs, but they can't communicate directly, so purely procedural programming wasn't possible. Messages were at the heart of the most complicated parts of the sneaker-buying event, but they also let me compartmentalize the process in a way that was easier to manage than it would have been otherwise, so building entirely around messages felt right.
Another nice feature of the background page: it can also inject arbitrary scripts into a page. These scripts execute in the context of the tab but with an important limitation: depending on how the site's scripts are defined, certain functions may not be visible. For instance, a page might have jQuery loaded, but my injected scripts might not be able to see
On FSB's Operation...
FSB targets footlocker.com's mobile site. Mobile sites tend to load faster than their desktop equivalents, so this seemed like the thing to pursue. I also noticed there's no CSRF token required, so you can add an item to your mobile cart from anywhere on the site without having to load the product page first! Dumb. Sadly, the mobile and desktop sites share a bit of code that causes some JS errors that screwed me up later on, but whatever...
So, with all that in mind, take a look at the code. It's very straightforward. I suggest installing the Chrome apps and extensions developer tool so you can monitor the background page. Examine each message received and sent through
background.js and you'll see how the process works. Each message injects a script, the script performs a task and sends a message back to the background page, which is received and starts another event. Messages can contain data and message receipt can return a callback to the sender. This let me capture data from an active tab, send it to the background page for collection, and/or request information from elsewhere. In all cases, the background page listens and responds to requests.
This really is the pattern repeated throughout the whole process: script is executed, tab sends message that script has started, background page returns data or instructions based on the message sent from the tab, tab receives message and processes the script's body, tab sends message when complete, the process completes. Call and response.
The readme at https://github.com/subvertallchris/fsb walks through operation, but I suggest you monitor each page through Chrome's console throughout the process. Background page, too. Each step announces its status. It WILL try to complete the purchase right now so if you don't want that, comment out the
click here. On the final page, the inspector was revealing some SSL problems — it looked like a requested resource didn't match its cert, so scripts would sometimes stop running. I'm sure there's a workaround but since my focus was on learning the extension, not being guaranteed to get a pair of sneakers, I didn't obsess over this detail.
There are two key areas that need work.
First, the UI of the page tab sucks. If I was trying to release this, I'd want something a bit friendlier, with a date/time picker and better size selection. I'd also want a more robust management console so you could see what was scheduled and what was active.
Second, it needs to make multiple simultaneous attempts to work around the weakness of Footlocker's servers. You'd need to manage the global state as well as the state of each tab.
I don't plan on going any further with this unless someone wants to help out. It was a fun project and I'm happy for the experience but I'm not looking to get any deeper into this nonsense. The money I've been spending on shoes is bad enough but I value my time even more and there are enough demands on that already!