Skip to content
April 15, 2015 / mb

A simple script to delete multiple keys from redis all at once

When developing, you sometimes want to delete a bunch of redis keys all at once. The KEYS command gets half of the job done, but what’s missing is a convenient way to wire it to the DEL command.

In these cases, (again, when developing, as the KEYS command is check-all-keys-in-database-slow) I often rely on this script:

#!/bin/bash
#
# A simple script to delete a bunch of keys from redis all at once.
#
# Don't use it in production!!!1!1!one

read -p "redis port to connect to? [6379] " redis_port

if [[ "${redis_port}" == "" ]]; then
    redis_port="6379"
fi

if [[ ${redis_port} =~ ^[0-9]+$ ]]; then
    while true; do
        read -p "keys to delete (wildcard accepted, e.g. user:foo:bar:*): " pattern
        # read all keys into a white space separated array
        IFS=$'\r\n' GLOBIGNORE='*' :; matches=($(redis-cli --raw -p ${redis_port} keys "${pattern}"))

        # list all matches
        echo "Matching keys:"
        matching_keys=0
        for line in ${matches[@]}; do
            echo $line
            matching_keys=$((matching_keys+1))
        done

        if [[ "${matching_keys}" -eq 0 ]]; then
            echo "No keys match your query, nothing to do."
        else
            read -p "Keys that will be deleted: ${matching_keys}. Really delete? [y/N] " confirm
            case ${confirm} in
                [yY]*)
                    echo -n "Keys deleted: "
                    redis-cli --raw -p ${redis_port} del "${matches[@]}"
                    ;;
            esac
        fi
    done
else
    echo 'must specify a valid port number'
    exit 1
fi

The nice thing about it is that it shows you what keys will be deleted before deleting them, so you have a chance of not screwing up your own DB! :)

Use it at will, but please, don’t use it in production!

Usage is straightforward, just chmod +x the script, and call it. Use Ctrl-C to exit the script.

January 15, 2015 / mb

Filter out spammers and click bait from Google Analytics

During the last few months, a new wonderful type of spam became part of my life: the Google Analytics spam.

As this article describes, what happens is that you start seeing some blatantly bogus traffic coming from a bunch of websites like semalt.com, buttons-for-website.com, darodar.com, or ilovevitaly.com.

Google announced an Automatic Bot and Spider filtering, but as some users on hacker news reported, it doesn’t work reliably.

So far, the only solution to this problem that worked for me is setting a filter, and add spammers to it as they come. There doesn’t seem to be that many as of today, so this approach is still usable.

To add a filter:

  1. go to your Administration page (last tab on your home page)
  2. All filters (on the leftmost column)
  3. New filter
  4. Choose Filter type “Custom” > “Exclude”
  5. Choose “Referral” from the Filter Field menu
  6. Set this as Filter pattern:
    semalt\.com|ilovevitaly\.co|priceg\.com|forum\..*darodar\.com|blackhatworth\.com|hulfingtonpost\.com|buttons-for-website\.com
  7. Select the views that you want to be filtered (I chose “All web site data”)
  8. Save

The filter pattern is a regular expression, so every time you find a new source of spam, simply add another “|spammersite\.com” (remember to escape dots with a backslash, as they mean “any character”).

It’s playing catch-up with spammers, but as long as Google doesn’t find a way to reliably detect them, it’s the only way to get rid of them. I’ve collected those 7 websites in a couple of months, and I’ve seen them being reported by other users as well. Since after setting the filter I’m no longer getting any bogus traffic, it looks like the problem is still relatively small and can be patched on case-by-case basis.

December 10, 2014 / mb

Chrome for Android freezes when scrolling

On my phone, scrolling web pages on Chrome recently became unbearably ugly. Every few seconds when scrolling Chrome will freeze completely, apparently no longer reacting to touch input, only to come back to life a split second later.

I haven’t seen this suggestion around, and it’s the only workaround that actually worked on my Samsung Galaxy Note III Neo.

Open chrome, and type chrome://flags in the address bar.

Look for the “Touch Scrolling Mode” setting (#touch-scrolling-mode), and set it to touchcancel.

On my phone, that fixed the scrolling problem completely, and the only drawback seems to be that I don’t have that fancy decelarated animation anymore. He, it never was that smooth to begin with, so I guess I’ll live with that :)

Hope it helps!

October 21, 2014 / mb

change single column collation in MySQL with ALTER TABLE

Ok, I may be a little rusty with SQL, but this took me surprisingly long to figure out.

If you want to change the collation for a single column in one of your tables, the correct syntax is

ALTER TABLE myTable 
MODIFY myColumn VARCHAR(width) CHARACTER SET charset COLLATE collation;

so for example, if you want to set your ‘id’ column in your ‘users’ table to accept case-sensitive ascii strings of length <= 10 characters, you would execute:

ALTER TABLE users
MODIFY id VARCHAR(10) CHARACTER SET ascii COLLATE ascii_bin;

Yes, it's easy, but I guess I was relying on google a bit too much this time (I could only find how to modify tables and databases, and many answers about columns in stackoverflow are just.. wrong!)

July 24, 2014 / mb

sudo asks for password even if NOPASSWD is set in /etc/sudoers

TL;DR when a script is not marked as executable and you try to run it with sudo, you don’t get the usual -bash: myScript.sh: Permission denied message, you are prompted for a password instead!

This one was very frustrating.

What I wanted to do was to make a user (let’s call him bran) able to execute a specific script (let’s call it /home/hodor/calm_down.sh) without having to provide his password, because the script will be executed by an automated tool (Jenkins).

I reached back to my earlier post about sudo, and updated the /etc/sudoers file so that its User privilege specification section looked like this:

root    ALL=(ALL) ALL
bran    ALL=(hodor)  NOPASSWD:  /home/hodor/calm_down.sh *

The last line gives user bran the ability to run /home/hodor/calm_down.sh as user hodor passing it any number of parameters (*) without having to provide his password (NOPASSWD:).

Saved it, su‘ed into bran, ran

bran@laketower:~$ sudo -u hodor /home/hodor/calm_down.sh "it's ok"

aaaaand…

[sudo] password for hodor: 

d’oh.

I checked the syntax in /etc/sudoers, and it was ok.

I checked whether any of the declarations that followed in /etc/sudoers could override the line I set for bran and hodor, none to be found.

Heck, I even put that line as the last line, so no line could override it. Nothing.

After a good hour of googling around and finding nothing, I remembered that the script is in a Git repository for which I just checked out a different branch. As it turned out, the script lost its executable bit.

So I set the executable bit again, as user hodor:

hodor@laketower:~$ chmod +x calm_down.sh
hodor@laketower:~$ logout
root@laketower:~# su - bran
bran@laketower:~$ sudo -u hodor /home/hodor/calm_down.sh "it's ok"
hodor.
bran@laketower:~$

it worked!

I’m sure there’s a legitimate security concern for this behavior, but dang! was this hard to figure out!

July 23, 2014 / mb

ForkedBooter steals window focus on Mac OS while Maven is running

This is about running Maven, specifically.

For most Java applications, any of the solutions from this question on StackOverflow should work.

It’s a matter of passing -Djava.awt.headless=true to the Java process, which can be done for all Java processes by saving that option to a variable named JAVA_TOOL_OPTIONS in your .bashrc.

That’s fine as long as you have direct control over the Java process, which is not the case with Maven.

In my case, it was the Failsafe Maven Plugin that caused that annoyance, but some were experiencing the same behavior with an old version of Surefire. The following snippet fixes both.

In your pom.xml, change your existing <plugin> definitions so that they look like this (the important lines are highlighted):

<!-- this is inside your <project><build><plugins> block -->
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>${maven.surefire.plugin.version}</version>
    <configuration>
      <includes>
        <include>**/unit/**/*Test*.java</include>
      </includes>
      <!-- prevent the annoying ForkedBooter process from stealing 
        window focus on Mac OS -->
      <argLine>-Djava.awt.headless=true</argLine>
    </configuration>
  </plugin>
  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>${maven.failsafe.plugin.version}</version>
    <configuration>
      <!-- prevent the annoying ForkedBooter process from stealing 
        window focus on Mac OS -->
      <argLine>-Djava.awt.headless=true</argLine>
      <includes>
        <include>**/integration/**/*Test*</include>
      </includes>
    </configuration>
    <executions>
      <execution>
        <goals>
          <goal>integration-test</goal>
          <goal>verify</goal>
        </goals>
      </execution>
    </executions>
  </plugin>

Done!

I added this reply to stack exchange as well, but maybe this post will be easier to find (as that question already already had an accepted answer when I replied, and it doesn’t work for Maven). We’ll see!

Follow

Get every new post delivered to your Inbox.

Join 92 other followers