Fix out of sync subtitles with Angular

In what seems to be a never ending series of posts on how to fix out of sync subtitles… here’s how to do it using Angular! ๐Ÿ™‚

First things first: here’s a link to the webpage that you can use to fix your out of sync subtitles. Just go there and follow the instructions, it’s really easy. Trust me, I’m the author ๐Ÿ˜›

A little background on why I made that page. About a year and a half ago I started toying around with AngularJS, because we decided to re-write the SketchTogether HTML client in it. Since I’m mostly taking care of the server side of the app, I spent a lot more time reading Angular code than writing it; even if I think I got the basic concepts relatively quickly, I wanted to have some first-hand experience with the library to really grasp what it means to develop an app with it, even if just a basic one.

Since back in those days I was fiddling with a toy python script to fix out of sync subtitles, I decided that it could have been fun to do the same thing in Angular. I wrote a rough implementation of the script’s functionality in about two weeks of my off time (about 10 very fragmented hours all in all, and I must say I was happy with it, considering how Angular is known for its steep learning curve), and left it there. Fast forward to 4 months ago, I decided to make it look a little prettier, and publish it on GitHub. And then, it took me 4 more months to find the time to write this blog post about it on a Sunday night ๐Ÿ˜›

My experience with Angular so far has been positive. For one thing, it’s not as hard to create a non-trivial webpage with it as most people want you to believe. Data binding can get you pretty far, and it’s relatively straightforward (one basic thing that left me wondering though is the from-dashed-to-camelCase directive name auto-conversion… that felt a bit like unneeded magic to me).

It does get a bit convoluted when your application grows and you want to add some piece of functionality that’s like 4 lines of code in jQuery, but which takes a new directive definition, changes to the HTML, and maybe a new scope, just to follow the Angular way. But then again, it could be my limited knowledge of the library (which I intend to keep expanding, because there’s so much to learn!).

As with most things Javascript, there tends to be an oversupply of sub-libraries to do pretty much anything. For subslider.js, for example, I found a lot of angular plugins to take care of pagination, with no clear winner. The same applies for drag-and-drop support (which in the end was implemented as a directive in the app). It really takes some time to simply understand which of the many plugins suits you best… just look at how many plugins are listed by ngmodules!

Oh, and one last thing. The “there should be a dot in your model” rule should be re-stated everywhere in the documentation. Once you really understand what it is about, you experience one of the major bumps in the famous “feelings about angular over time” graph from this post ๐Ÿ™‚

Filter out spammers and click bait from Google Analytics

During the last few months, a new wonderful type of spam became part of my life: the Google Analytics spam.

As this article describes, what happens is that you start seeing some blatantly bogus traffic coming from a bunch of websites like semalt.com, buttons-for-website.com, darodar.com, or ilovevitaly.com.

Google announced an Automatic Bot and Spider filtering, but as some users on hacker news reported, it doesn’t work reliably.

So far, the only solution to this problem that worked for me is setting a filter, and add spammers to it as they come. There doesn’t seem to be that many as of today, so this approach is still usable.

[Update – July 2015]: if you have a public HTTP/PHP server available, and are willing to invest half a day to install it, piwik is a nice free, open-source Google Analytics alternative. Piwik uses a community-maintained list of spammers that can also be used in Google Analytics. They wrote a blog post about it, too.

I’ve been using piwik for a few weeks now, and I’m happy with it so far. The nice thing is that updates are very easy to apply, and they include the most recent list of spammers available. The thing that could be improved is the installation process, it’s not as easy as it could be (at least if you’re using Nginx as web server). They also have a Cloud-hosted version, but I guess that if you’re using Google Analytics for free, you’re more interested in free alternatives!

To add a filter in Google Analytics:

  1. go to your Administration page (last tab on your home page)
  2. All filters (on the leftmost column)
  3. New filter
  4. Choose Filter type “Custom” >ย “Exclude”
  5. Choose “Referral” from the Filter Field menu
  6. Set this as Filter pattern:
    semalt\.com|ilovevitaly\.co|priceg\.com|forum\..*darodar\.com|blackhatworth\.com|hulfingtonpost\.com|buttons-for-website\.com
  7. Select the views that you want to be filtered (I chose “All web site data”)
  8. Save

The filter pattern is a regular expression, so every time you find a new source of spam, simply add another “|spammersite\.com” (remember to escape dots with a backslash, as they mean “any character”).

It’sย playing catch-up with spammers, but as long as Google doesn’t find a way to reliably detect them, it’s the only way to get rid of them. I’ve collected those 7 websites in a couple of months, and I’ve seen them being reported by other users as well. Since after setting the filter I’m no longer getting any bogus traffic, it looksย like the problem is still relatively small and can be patched on case-by-case basis.

Chrome for Android freezes when scrolling

On my phone, scrolling web pages on Chrome recently became unbearably ugly. Every few seconds when scrolling Chrome will freeze completely, apparently no longer reacting to touch input, only to come back to life a split second later.

I haven’t seen this suggestion around, and it’s the only workaround that actually worked on my Samsung Galaxy Note III Neo.

Open chrome, and type chrome://flags in the address bar.

Look for the “Touch Scrolling Mode” setting (#touch-scrolling-mode), and set it to touchcancel.

On my phone, that fixed the scrolling problem completely, and the only drawback seems to be that I don’t have that fancy decelerated animation anymore. He, it never was that smooth to begin with, so I guess I’ll live with that ๐Ÿ™‚

Hope it helps!

How long does it take for the Google Play Store to publish my app in beta?

Your mileage may vary, but it shouldn’t take significantly longer than a couple of hours. If you’re not too unlucky, it should probably take just one hour, both when you’re publishing an app for the first time, and when you’re releasing an update for an existing app.

If it’s taking significantly longer than that (I’m talking more than one day here), you may have been fooled by Google’s peculiar requirements for deployments.

This fooled me twice (so I guess, shame on me).

When you publish an app for the first time, you need to create a Google Group or a Google+ community for testers before the link to your app becomes available. If you try to access the link (even using your own Google account) without being a member of the configured group/community, you’re greeted with a 404 page.

Last time I published an app though, setting an existing Google Group for testers (one that I had configured for a different app) didn’t seem to work. When I created a new Community on Google+, it worked right away. I don’t know if I was just unlucky or if you need a dedicated Google Group for every app, but if you still get a 404 after setting up the Google Group, just try with Google+.

After you’ve become a tester for the app (by visiting your app’s page on the Play Store) it usually takes another hour or so before the APK becomes available for download.

Create a list of movies to watch with Python and Urlist

I usually like to keep lists of movies to watch, books to read, games to play on Google Keep, mainly because it comes with a widget that looks nice on my phone. Its sharing capabilities though are, well, nonexistent. Yes, you can email a list with a bullet point for each entry but it ends there.

Since I wanted to share a list of movies to watch with my wife, I resorted to Urlist. It’s a neat, straight-to-the-point tool that’s good for sharing links with friends, collaborators, anybody.

I wish they had an API available, so that this post could have been about a tool that automatically creates lists for you (heck, I could even write a simple chrome extension!), but so far there’s none.

Our list is going to have, for every entry, the vote that the movie got on IMDB, a brief summary of its plot and cast. If any of them attracts your SO’s attention, (s)he can just click to see further info about the movie ๐Ÿ™‚

The script requires BeautifulSoup and Requests, 2 awesome libraries to scrape the web.

To install them, you can use either pip:

sudo pip install beautifulsoup4 requests

or easy_install:

sudo easy_install beautifulsoup4 requests

Create the list on Urlist, launch the script:

python scrape_IMDB.py

and for every movie you want to add:

  1. search it on IMDB
  2. copy the URL
  3. paste the URL on Urlist to add an entry
  4. paste the URL on the console where the script is running
  5. copy the output of the script
  6. back to Urlist, hit edit and paste what you copied

Here’s the script (you can download it from pastebin):

from bs4 import BeautifulSoup
import requests

done = False

while not done:
  try:
    url = raw_input("IMDB URL: ")

    # get the IMDB page
    r = requests.get(url)
    data = r.text

    # and parse it with BeautifulSoup
    soup = BeautifulSoup(data)

    # the td containing what we're looking for
    td = soup.find('td', {'id': 'overview-top'})
    rating = td.find('div', {'class': 'star-box-giga-star'}).string
    plot = td.find('p', {'itemprop': 'description'}).string
    # the div containing the main actors in the cast
    actors = td.find('div', {'itemprop': 'actors'})
    stars = ', '.join([actor.string for actor in actors.find_all('span', {'class': 'itemprop', 'itemprop': 'name'})])

    print '*%s* - %s. %s' % (rating.strip(), stars, plot)
  except KeyboardInterrupt:
    done = True
print
print 'bye!'

It’s super simple! It gets the page, finds the HTML source for what we’re looking for, and prints it out as formatted text that’s good for Urlist.

The way you find items with BeautifulSoup is relatively similar to what you do with jQuery: you look for elements in the DOM that contain what you’re looking for (to find what they are, just use your browser’s inspector… on Chrome, right click on the text and choose “Inspect element…” to see where it is in the DOM), and manipulate them as strings or arrays of strings.

Easy enough!

Logging in Javascript and filtering by tag

I’m not a fan of loggers in general when it comes to simplicity. They all require you to spend (waste) some time on understanding their configuration syntax, maybe dealing with XML files (what am I, a caveman? :P), and they often require you to adapt to their philosophy. By that I mean, there’s plenty of questions on StackOverflow similar to this, and I definitely share this guy‘s feeling about the issue.

In Javascript, I’ve seen some libraries for logging, and they all look quite complicated to set up. Or, quite complicated considering it’s Javascript we’re talking about. JSLog doesn’t require that much configuration, but most of the times I don’t even care about log levels in JS “apps”, I just want to filter messages according to their context. So I may be interested in all ajax-related logs at some point, but not in DOM manipulation messages. At some other time, I may be interested in DOM manipulation logs, but not in ajax-related messages. These can be though of as log tags (the “DOM” tag, the “ajax” tag, the “user-input” tag, and so on).

A quick and dirty way to deal with all of this is to define some Log functions named after the “context” in which they’re called, and assign them to a no-op function when you want to filter them out.

In Chrome, these could be the log functions (let’s say they’re stored in log.js):

function Log() {
}

Log.message = function(message) {
    console.log(message);
};

Log.dom = Log.ajax = Log.message;

Then, throughout your code, you may have calls like:

Log.dom('adding sidebar');
// in some other place...
Log.ajax('new msg received from server');

All you need to do to disable e.g., all ajax logs, is just change the Log.ajax function to a no-op in log.js, like this:

Log.ajax = function(){};

you can even do it live using the Chrome Developer Tools!

So a question could be: aren’t those calls to a no-op function expensive? Javascript function calls are expensive, right? Why not add a flag like this:

if (logAjaxEnabled) {
   Log.ajax('new msg received from server');
}

to all log calls?

Well, it’s a pain to add all those checks, that’s why ๐Ÿ™‚

And your mileage may vary, but in all browsers I’ve tested, calling a no-op is not that bad. It’s way worse than using the flag, but it’s in the same ballpark (assuming you’re not writing some very CPU intensive code), and it’s WAY less painful to use, cause you don’t need to write all those if statements.

I put together a dumb benchmark on JSPerf to see what’s the performance drawback for using this method, you can try it!

It’s also interesting to note how browser performances for something like this vary that wildly.. ah, the JS world!

I’m sure I’m not discovering anything here (I mean, just look at the first few links for a simple Google query like this), but I haven’t seen this approach being used that much (filtering by context, and not by log level). It’s something very close to what Android’s LogCat does, and I really like it!