Extremism and Inclusion in the Metal Underground

A well-known-in-the-underground, extremely liberal band called Caïna were kicked from a German music event for covering the band Death in June in the past. You can read their statement about it here.

This situation is sad for many reasons. Among them, it’s a reminder that many people view others in very black and white, static terms, where reflection, discussion, and growth are impossible once any connection to “shady” ideas or people have been made. They see Death in June’s Douglas P as someone who, despite owning his mistakes as the actions of a young person trying to find his way in the 80s, will be forever tainted. Worse still, they see someone like Caïna’s Andy Curtis-Brignell as equally untouchable for agreeing that Douglas could grow up and should not be punished for mistakes he is trying to move past. These are the same people who reject reformed skinheads from participating or object to bands containing minorities have been photographed wearing shirts that they find questionable.

Call me crazy, but I think that the most powerful advocates for social change in metal are people with connections to things that make us uncomfortable. I have more than a few friends who had terrible opinions when they were young but grew out of them. The keys to this transformation: the opportunity to relate to people with different backgrounds and shared interests. The time and place to realize that their attitudes just don’t make sense.

Nothing erodes racist attitudes like finding common ground and building connections with different types of people. People who go through these transformations understand how others end up where they were. Their inclusion in the underground makes the statement that it’s possible to change. Their presence is crucial, even if what they contribute in the fight against extremism happens slowly and without any deliberate actions. The wholesale rejection of individuals who at some point engaged in actions we find questionable makes it harder, sometimes impossible, to reach those who need common ground the most. This goes for folks who grow out of terrible perspectives and those of us who are willing to accept change in others.

But back to Caïna and this specific event, Andy is exactly the type of person we need more of in the underground. Very few black metal artists are willing to make strong statements against discrimination – he is one of those few. He probably also has a surprising number of listeners who disagree with his views, which means that his music can often present opportunities for communication between people who would otherwise have trouble finding common ground. To prohibit his music and his fans from participating in an event because he chooses to believe that people can change, that art can unite, is a sad, sad decision with profound implications.

I think about this whole thing and wonder if there is someone somewhere who actually does have right-leaning tendencies and will take this as one more reason to feel persecuted and isolated. They’ll say that this is more “SJW PC nonsense,” they’ll be more willing to side with actual skinheads when they are kicked off of a show for actually being shitheads, they’ll make shittier friends, and they’ll fall deeper into this pit. I’m not saying that this one incident is the start of a slippery slope – I don’t see how it could be – but it contributes to a worldview that is best fought through inclusion and dialogue, not exclusion and shame.

Instance Variables, Methods, and the Public API

A post the other day on /r/ruby asked about the reasoning behind the recommendation that instance variables be accessed through methods instead of directly throughout code where they are in scope. Most of the responses dealt the fact that this helps make your code more maintainable, which is true, but I think there’s a more significant reason for this: it adds data to your class’s public interface, so it encourages healthy design habits and forces some introspection.

In a carefully designed class, changes to interface are taken seriously. Before you add, remove, or change it, you should ask some questions, the most important (in my opinion) of which are, “Is this method reasonable, given the stated purpose of this class?” and “Is this a method something that users can trust to be there and work properly in future releases of this library?” When you expose an instance variable as a method, you implicitly respond to both of those statements in the affirmative, so think carefully about whether those questions are also true of that instance variable!

When you find yourself in the position that you have an instance variable that shouldn’t be exposed as part of the public API, consider that you might have a class with too much responsibility or an instance variable that you don’t need. Unneeded instance variables often come from doing this:

def my_process

def get_foo
  @foo = ObjectLookup.find_by(quality: 'foo')

def use_foo_here
  @bar = @foo + 10

def use_foo_again
  @baz = @bar + 20

def return_value
  @bar > 40 ? @bar : @foo

…when something like this is healthier in every way:

def my_process
  foo = get_foo
  bar = plus_10(foo)
  baz = plus_20(bar)
  return_value(foo, baz)

def get_foo
  ObjectLookup.find_by(quality: 'foo')

def plus_10(obj)
  obj + 10

def plus_20(obj)
  obj + 20

def return_value(starting, ending)
  ending > 40 ? ending : starting

Yes, there are times whenyou want to reuse an expensive query multiple times internally within an object and exposing it to the public API would be inappropriate. That’s fine, just mark that method private. But that should be your last result, not an excuse for extra data. Changes to the public API are an opportunity to question your approach and improve the quality of your code, so face them head on!

Now with 100% Less WordPress

Since 2011, this blog has been powered by WordPress. As of today, it’s entirely static and generated by Jekyll. Isn’t it funny how things have changed?

WordPress is a very cool product. It does so many things so well and has managed to dominate such a huge portion of the web that it demands respect. I can sum up my feelings about it pretty easily:

  • The templates and plugins system is great, mostly.
  • The basic WYSIWYG editor is fantastic until you discover Markdown.
  • It’s easy to update your installation and your plugins, except when it’s not.
  • Update recommendations help keep you safe, as long as you check frequently.

In other words, its strong points are really strong but its weak points – its risky plugins, the horror that is troubleshooting when something breaks, the need to constantly be on top of security as a result of its huge API – are pretty big for someone who just wants a place to talk shit every few weeks or months.

Sometime last year, we launched an official site for Neo4j.rb, Neo4jrb.io, and decided to host through Github.io and use Middleman. It’s been wonderful. My big takeaways from a year-ish of working with a statically generated blog:

  • Local dev and Git deployment > remote Admin page. It makes writing as convenient as coding.
  • Vim and Markdown > WYSIWYG editor. It makes writing as fast as coding.
  • We essentially don’t have to worry about security.
  • Modifying HTML and CSS > modifying themes.

It was kind of a no-brainer. I wanted something to help me write more often and this seemed like a good excuse, so I took the plunge. Even though Github.io is free, I decided to self-host because I like having an SFTP server and space to experiment. A search revealed a WordPress plugin to migrate content straight to Jekyll, so that made the platform decision easy. Ultimately, migration was simple enough:

  • Basic new server setup: nginx, users, updates, etc,…
  • Export old content using this plugin
  • Install rbenv, Ruby, Jekyll and related gems
  • Install git and configure post-receive hook, mosty descrived here
  • Tweaks to migrated new blog: styling, fixing some things the migration didn’t get right, install/configure jekyll-archives

It took most of a day, when said and done, but none of it was what I would describe as “hard.” There are still some styling tweaks to make but I’m happy with it, overall. The real hard part comes next: actually writing more often.

Sneakerbots, Chrome Extensions, and the Evented Model

I like Air Jordan 1s. A lot. It's a new thing, especially since I hadn't owned a pair of shoes that wasn't all black since 1998, probably, but I'm enjoying this opportunity to step outside of my comfort zone and try something different.

Did you know that "sneakerbots" are a thing? Cause they are a thing. Limited edition sneakers are announced ahead of time and like any limited, premium item, there is a market around their buy and sell, so, naturally, a market has emerged around the very means to buy them in the first place. I watched a few YouTube videos and from what I can tell, "sneakerbots" look like applications that watch the clock and then zoom through the checkout process as soon as the shoes go on sale. The kicker is that few of them are free. Buyers often pay $25-$75 for a license that claims to work with all of the big retailers' websites.

I've been reading /r/sneakers a lot lately and there seems to be some misunderstanding about how they work. In particular, it seems like a lot of people have this idea that sneakerbots are some magic tools that guarantee a successful purchase, but the truth is that they are limited by the web sites' servers' abilities to process requests. This, ultimately, is the greatest weakness of these things because it means that no matter how good your script is, you're always at the mercy of the server's ability to handle traffic. At the same time, it stands to reason that traffic will ramp up in the milliseconds after a pair goes on sale, so the faster you can hit submit, the better your chance of completing a purchase.

Regardless of whether paid sneakerbots are worth it, the whole thing bugs me. It's fucked up that most new releases are only accessible through eBay at 3x or more their original price. It's fucked up that I can buy new releases from literally hundreds of different auctions for $500+ but I've not once seen a pair on someone's feet in Manhattan or Brooklyn. These new releases are going straight from the shelves to eBay and I'm guessing that most will never see concrete. The paid sneakerbot market adds insult to injury, exploiting the desire to catch a drop without being able to guarantee a sale. The videos I saw looked shady as shit: you're installing some closed source software built anonymously and designed to exploit shopping carts. It has access to your browser, your account, probably your credit card info. Fuck everything about that.

Enter FSB, my open-source sneaker buyer in the form of a Chrome extension. I had been looking for an excuse to learn a bit more about extensions anyway and if I can give an alternative to paid shady software, that's cool. (I guess it'd be better if it was a bit easier to use, but whatever.) My current version is shitty: it targets Footlocker's mobile site, it only works with shoes, and there might be a bug that's keeping the checkout button from being clicked. (In fairness, it's a bug caused by Footlocker's mobile site, a mismatched cert name that stops JS execution.) I threw it together in a bit more than a day and when I tried to use it the morning of the Shattered Backboards release, Footlocker's shitty servers were a show-stopper. At the end of this post, I'll talk about how it could be improved and what it would take to build a better sneakerbot. You can find the Github repo at https://github.com/subvertallchris/fsb.

Failure aside, the process of building a simple Chrome extension was fun and more valuable than a pair of sneakers would have been. The most interesting part of the process was figuring out how to create a concept of shared state using the extension's background page. I ended up with a pattern reminiscent of an evented model, something we've been using quite a bit at work with Ruby and RabbitMQ, but entirely in Javascript with the some help from the Chrome extensions API.

I'll walk through some of the things that took time to figure out.

Technical Overview

Start by reviewing the docs on extension setup, I won't cover that since there are many good tutorials out there for that.

My manifest.json likely demands more permissions than it. It uses a non-persistent background page to maintain state, listen for events, and fire off new pieces of the process as events occur. cartwatcher.js and cartcheckout.js both act as registration scripts. They ping the background page so it can record their tab id and, if needed, start a process when it discovers them.

Chrome's extensions API makes it very easy to manipulate CSS and start scripts, but if you want to, say, get the value of a field from a form and use it somewhere else, or make the clicking of a button on tab 1 cause a button on tab 2 to be clicked, it can be tricky. background.js contains the bulk of the code that solved this problem and, mentioned above, feels a lot like how we use RabbitMQ to kick off actions on servers at work.

Evented Programming for Dummies

Most programmers, myself included, typically approach tasks from a procedural perspective. We think in terms of sequence: first you do A, then B, then C, and so on. In the Ruby language, we have something like this:

def a_method

def do_this
  # some things happen

def then_this
  # some more things happen

def finally_this
  # final things happen

An evented model decouples one action from another. Instead of defining the sequence, you create independent actions that announce their behavior through messages, or events. Each action can listen for and respond to others. Think of it as, "When Section A completes, it announces, 'SECTION_A_DONE'." Anything that is listening for "SECTION_A_DONE" will start working on its own. Section A neither knows nor cares what happens after it and those actions that start don't care what fired the message, they just know that it's their turn to do something. We also do away with the idea of a central function that ensures that Section A flows into Section B. At my day job, we use a server called RabbitMQ to manage this process and some programming languages, like Go, have messaging baked in as core concepts.

With FSB, I had a few different tasks happening independently of one another: I needed a cart open, I needed something else watching a timer and prepared to POST a form, I needed to listen for a checkout and complete the purchase when the cart was populated. There are some procedural elements to it but approaching it from an evented perspective let me think in terms of state, not process. This worked well with Chrome's Extensions API. Tabs are able to send messages to the persistent background page, and the background page can send messages back to tabs, but they can't communicate directly, so purely procedural programming wasn't possible. Messages were at the heart of the most complicated parts of the sneaker-buying event, but they also let me compartmentalize the process in a way that was easier to manage than it would have been otherwise, so building entirely around messages felt right.

Another nice feature of the background page: it can also inject arbitrary scripts into a page. These scripts execute in the context of the tab but with an important limitation: depending on how the site's scripts are defined, certain functions may not be visible. For instance, a page might have jQuery loaded, but my injected scripts might not be able to see $. In the case of FSB, this was a bit of a problem — I wasn't able to call their function to move onto the cart, so I had to use JavaScript to click the button that kicked off that process. At one point, I inject jQuery along with my script to perform an AJAX POST.

On FSB's Operation...

FSB targets footlocker.com's mobile site. Mobile sites tend to load faster than their desktop equivalents, so this seemed like the thing to pursue. I also noticed there's no CSRF token required, so you can add an item to your mobile cart from anywhere on the site without having to load the product page first! Dumb. Sadly, the mobile and desktop sites share a bit of code that causes some JS errors that screwed me up later on, but whatever...

So, with all that in mind, take a look at the code. It's very straightforward. I suggest installing the Chrome apps and extensions developer tool so you can monitor the background page. Examine each message received and sent through background.js and you'll see how the process works. Each message injects a script, the script performs a task and sends a message back to the background page, which is received and starts another event. Messages can contain data and message receipt can return a callback to the sender. This let me capture data from an active tab, send it to the background page for collection, and/or request information from elsewhere. In all cases, the background page listens and responds to requests.

This really is the pattern repeated throughout the whole process: script is executed, tab sends message that script has started, background page returns data or instructions based on the message sent from the tab, tab receives message and processes the script's body, tab sends message when complete, the process completes. Call and response.

The readme at https://github.com/subvertallchris/fsb walks through operation, but I suggest you monitor each page through Chrome's console throughout the process. Background page, too. Each step announces its status. It WILL try to complete the purchase right now so if you don't want that, comment out the click here. On the final page, the inspector was revealing some SSL problems — it looked like a requested resource didn't match its cert, so scripts would sometimes stop running. I'm sure there's a workaround but since my focus was on learning the extension, not being guaranteed to get a pair of sneakers, I didn't obsess over this detail.

Improving FSB

There are two key areas that need work.

First, the UI of the page tab sucks. If I was trying to release this, I'd want something a bit friendlier, with a date/time picker and better size selection. I'd also want a more robust management console so you could see what was scheduled and what was active.

Second, it needs to make multiple simultaneous attempts to work around the weakness of Footlocker's servers. You'd need to manage the global state as well as the state of each tab.

I don't plan on going any further with this unless someone wants to help out. It was a fun project and I'm happy for the experience but I'm not looking to get any deeper into this nonsense. The money I've been spending on shoes is bad enough but I value my time even more and there are enough demands on that already!

Model Caching in Rails, or when a Student is not a Student

For a few months, we’ve had a few reports by users from Neo4j.rb users of an odd bug. The story goes, “I try to create a relationship between two nodes but the type checking tells me that one of the nodes is not of the appropriate type but I know that it is.” In code, it could look like this:

student = Student.first
lesson = Lesson.first
# Creates a relationship between the lesson and student
lesson.students && student

More specifically, it was always reported by Rails users dealing with create actions. They’d be loading a node, creating a new node of another class, and associating the new node with the old.

student = Student.create(student_params)
lesson = Lesson.find(lesson_id)
lesson.students && student

An error would be raised saying words to the effect of, “Node type invalid. Expected , found ."

In my arrogance, I assumed that since the errors were not reported that often and I had never personally seen it, these guys must have had some conflict with another gem or messed up code somewhere. After all, if it was a real bug, we’d have seen it ourselves or our tests would have caught it, right? No, wrong — completely wrong. Finally, someone copied/pasted their code to me and I could see everything was done correctly. They also noticed that it only happened after updating a file and restarting their server would fix it. It began to sound like an issue with the Rails automatic reloader, so I set out to track it down.

Classes are Still Instances

The root, or maybe roots, of the problem turned out to be variables and constants hanging onto copies of classes, persisting across Rails reload!. There were two spots where this was happening, but to understand why it’s a problem, we need to talk about what a class is.

In Ruby, a class is an instance of Class. Because of that, it is possible that active Ruby process can have two independent instances of the same class, one a different version of the other. It’s something like this:

container = Container.new
container.enclosed_model = Student
# the Student model is changed, Rails `reload!` is called
# From this point out, our `Student` model can be thought of as Student v2.

# This is an instance of Student v2
student = Student.new

# This is false because our container's `enclosed_model` is Student v1
# =>false

# And this hints at why
Student.object_id == container.enclosed_model.object_id

Comparing two objects’ object_id results is always the final answer on whether they are the same. By spitting out the object ids before the error was raised, I saw that, yes, Rails was loading a new instance of the class but somewhere in the bowels of Neo4j.rb was a reference to the old version.

After that, there were two questions remaining: where were references to old models hanging around and how could I keep things current?

The Culprits

The answer to the first question was pretty easy to track down but might be a bit too specific to Neo4j.rb to warrant inclusion in this blog post so I’ll just go over it briefly.

Exhibit A: Node Wrapping

When nodes are returned from the database, we have to figure out which model is responsible for it, and doing that requires us to find the model that has the same combination of labels as the node. It’s not a terribly expensive process but it can be wasteful since you only really need to perform that process once, when a combination of labels is first encountered, and should be able to reuse the results every time after. The results are stored in a hash, MODELS_FOR_LABELS_CACHE, which maps labels => model_constant. In a normal Ruby app, it’s not a big deal that this never gets GCed since you will only have so many possible entries; in Rails development mode, with models constantly being reloaded, that’s not always true, so it was possible for node’s to be instantiated using the wrong versions of models!

Exhibit B: Association models

The associations system and QueryProxy class are arguably the coolest features of Neo4j.rb. You’re able to define associations in models on the fly. Once defined, you can easily create and traverse relationships. It’s possible to build relatively complex queries that jump between models, executed lazily, as you do in ActiveRecord. What makes Neo4j.rb’s association chaining cool is that it will only perform one query, no matter how many models you jump across. So this:


…will find the lessons of children whose parents are teachers who teach classes taken by one student. In Cypher, it could look something like:

MATCH (s:Student)-[:ENROLLED_IN]->lesson)-[:TAUGHT_BY]->teacher)-[:PARENT_OF]->child)-[:ENROLLED_IN]->result) WHERE ID(s) = {student_id} RETURN result

In a model, the association definitions look like this:

has_many :out, :lessons, type: 'ENROLLED_IN'

That method returns an instance of Neo4j::ActiveNode::HasN::Association, which becomes bound to the model in its @associations hash. This association instance needs to know the model on the other side, so whether you let it infer that from the association name or use model_class to set it, it ends up with a @model_class instance variable that stores — you guessed it — the constant of the other class. When you try to create a relationship, it does a type check in Ruby. If the node given is not an instance of the model saved in @model_class, it raises an error. And there we have it: lesson.students && student will raise an error if student was not borne of the same version of Student held in its @model_class.

Whew! So… now what?

Learning to Let Go… of Models

The answer to the second question, how do we clean things up, was found just last night.

When I first diagnosed the problem, I was eager to get a workaround in so I patched the gem to not use is_a? to determine association compatibility, opting instead to use the node’s labels compared to the model’s labels. This solved the immediate problem but it wasn’t a real fix. Last night, someone commented to an issue on the devise-neo4j library that I had forgotten about, describing the same problem, and I realized that there’s a very good chance this caching was the root. He had done some research into Devise and posted a snippet that included a reference to ActiveSupport::Dependencies::ClassCache, so I looked that up and found a note about the before_remove_const method.

before_remove_const seems to be the solution. When implemented as a class method, it is called by Rails reloader at the start of a reload cycle. I was able to use it to wipe out the constants that were hanging onto models and trigger a refresh of @model_class in each association. You can see the PR here. I say it “seems” to be the solution because I’m still waiting on confirmation of the devise-neo4j issue’s resolution, but I’m reasonably confident. Even if it doesn’t, I think we’ve confirmed that there’s an old reference to a model hanging out somewhere, so we just have to figure out what we missed and queue it for update later on.

So there you have it! An interesting bug squashed and in the process, we saw more proof of Ruby’s “everything-is-an-object” ethos. We learned a bit more about ActiveSupport, some best practices when caching class names, and a crucial reminder to take bug reports seriously, even if they seem impossible to you.

subscribe via RSS