Skip to content
August 7, 2015 / mb

Find the most recent file/folder in a folder with Java 8

A simple snippet that shows how Java 8 lambdas can be really nice to replace operations that used to require a lot of boilerplate in earlier versions of Java.

If you want to find the most recent file or subfolder in a folder with Java 8, here’s what you need to do:

Path parentFolder = Paths.get("path", "to", "your", "file");

Optional<File> mostRecentFileOrFolder =
    Arrays
        .stream(parentFolder.toFile().listFiles())
        .max(
            (f1, f2) -> Long.compare(f1.lastModified(),
                f2.lastModified()));

if (mostRecentFolder.isPresent()) {
    File mostRecent = mostRecentFileOrFolder.get();
    System.out.println("most recent is " + mostRecent.getPath());
} else {
    System.out.println("folder is empty!");
}

The very nice thing is that you can take advantage of the flexibility of Java 8 streams to either make the operation parallel (just throw in a parallel() call after Arrays.stream()), or to filter results according to other criteria.

For example, if you’re only interested in one type of child elements (a file or a folder), you could…

// if you're only interested in files...
Optional<File> mostRecentFile =
    Arrays
        .stream(parentFolder.toFile().listFiles())
        .filter(f -> f.isFile())
        .max(
            (f1, f2) -> Long.compare(f1.lastModified(),
                f2.lastModified()));

// if you're interested in folders...
Optional<File> mostRecentFolder =
    Arrays
        .stream(parentFolder.toFile().listFiles())
        .filter(f -> f.isDirectory())
        .max(
            (f1, f2) -> Long.compare(f1.lastModified(),
                f2.lastModified()));
July 9, 2015 / mb

Getting the Git commit id of a Gerrit Change Set

Oddly enough, Gerrit doesn’t report the Git commit id of its Change Sets.

The only way I found to get that piece of information is to use its SSH interface, using the query command.

The syntax is this:

ssh -p 29418 user@gerritHost gerrit query --current-patch-set <ChangeId>

replacing 29418 with the port you normally use to contact Gerrit via SSH.

That will display a number of properties about the Change Set, including the revision parameter which is the extended version of the Git commit id.

You can use the --format JSON option if you want to get a JSON representation of the same data, or you can get the short commit id using for example:

ssh -p 29418 user@gerritHost gerrit query --current-patch-set <ChangeId> \
cut -d':' -f2 | cut -c2-7
July 8, 2015 / mb

List forwarding rules in VirtualBox running headless (even with older VBoxManage versions)

The VBoxManage command is very handy for managing virtual machines in servers running headless. Adding and removing forwarding rules is super simple, you just call for example

# adding "rule1" for "vm_1" mapping TCP host_port to guest_port
VBoxManage modifyvm vm_1 --natpf1 "rule1,tcp,,host_port,,guest_port"

# deleting that same rule
VBoxManage modifyvm vm_1 --natpf1 delete rule1

I created two simple scripts so that I don’t have to remember the exact syntax every time: vm_forward_add and vm_forward_delete. You run the commands with no arguments, and they tell you what you need to add.

Listing the existing rules is a bit more complex, as VBoxManage has no option for that (not the version that I use, at least).

This is where xmlstarlet comes to the rescue. Xmlstarlet is a command-line tool for parsing XML that comes pre-installed in most UNIX systems.

Since forwarding rules are stored in VirtualBox .vbox files, which are just big XML files storing all properties for a VM, we can grab the information we want from there.

This is the script that I use to list all forwarding rules. I can’t guarantee that it works with any version of VirtualBox, but it works with any version <=4.3 (the one I use).

Feel free to use the script, the worst that it can do is… not work :)

About things that weren't too obvious: you need to include the XML schema not only in the XPath of the XML node you're interested in, but in all definitions (!).

So, say you have a node with this path and these attributes (xmlstarlet el -a lists all XPaths available in the XML file):

me@server:~ xmlstarlet el -a my_vm.vbox | grep Forwarding
VirtualBox/Machine/Hardware/Network/Adapter/NAT/Forwarding
VirtualBox/Machine/Hardware/Network/Adapter/NAT/Forwarding/@name
VirtualBox/Machine/Hardware/Network/Adapter/NAT/Forwarding/@proto
VirtualBox/Machine/Hardware/Network/Adapter/NAT/Forwarding/@hostport
VirtualBox/Machine/Hardware/Network/Adapter/NAT/Forwarding/@guestport

To query those four attributes you need to specify the XML schema using the -N option (easy to find in xmlstarlet‘s docs), but you also need to set that schema for every single attribute name!

So, this will fail:

xmlstarlet sel -N x="http://www.innotek.de/VirtualBox-settings" -t \
-m "//x:Machine/Hardware/Network/Adapter/NAT/Forwarding" -v @name \
-o " " -v @hostport -o " " -v @guestport -n my_vm.vbox

but this will work (note the repeated x:):

xmlstarlet sel -N x="http://www.innotek.de/VirtualBox-settings" -t \
-m "//x:Machine/x:Hardware/x:Network/x:Adapter/x:NAT/x:Forwarding" \
-v "concat(@name, ' - host:', @hostport, ' guest:', @guestport)" \
-n my_vm.vbox

I wish this was more explicit in xmlstarlet documentation!

July 6, 2015 / mb

openfire: fix “there was an error one importing private key and signed certificate”

This took me frustratingly long to find out.

When importing a signed SSL certificate on my openfire server, the web interface would always show me this error:

there was an error one importing private key and signed certificate

Typo aside, that’s some pretty obscure error.

Users have come up with detailed guides on how to convert certificates to other formats, setting system properties to make them work once imported back, but… Nothing helped, in my case.

The solution to my problem was: when you import your certificates (using the form in the Server Settings/Server Certificates section), just… type something on the Pass Phrase used for creating Private Key. Even if you never set a password, type something, anything will do. Just don’t leave it empty.

I’m talking about this guy:

openfire

That’s it, you can restart your server. Meh.

April 15, 2015 / mb

A simple script to delete multiple keys from redis all at once

When developing, you sometimes want to delete a bunch of redis keys all at once. The KEYS command gets half of the job done, but what’s missing is a convenient way to wire it to the DEL command.

In these cases, (again, when developing, as the KEYS command is check-all-keys-in-database-slow) I often rely on this script:

#!/bin/bash
#
# A simple script to delete a bunch of keys from redis all at once.
#
# Don't use it in production!!!1!1!one

read -p "redis port to connect to? [6379] " redis_port

if [[ "${redis_port}" == "" ]]; then
    redis_port="6379"
fi

if [[ ${redis_port} =~ ^[0-9]+$ ]]; then
    while true; do
        read -p "keys to delete (wildcard accepted, e.g. user:foo:bar:*): " pattern
        # read all keys into a white space separated array
        IFS=$'\r\n' GLOBIGNORE='*' :; matches=($(redis-cli --raw -p ${redis_port} keys "${pattern}"))

        # list all matches
        echo "Matching keys:"
        matching_keys=0
        for line in ${matches[@]}; do
            echo $line
            matching_keys=$((matching_keys+1))
        done

        if [[ "${matching_keys}" -eq 0 ]]; then
            echo "No keys match your query, nothing to do."
        else
            read -p "Keys that will be deleted: ${matching_keys}. Really delete? [y/N] " confirm
            case ${confirm} in
                [yY]*)
                    echo -n "Keys deleted: "
                    redis-cli --raw -p ${redis_port} del "${matches[@]}"
                    ;;
            esac
        fi
    done
else
    echo 'must specify a valid port number'
    exit 1
fi

The nice thing about it is that it shows you what keys will be deleted before deleting them, so you have a chance of not screwing up your own DB! :)

Use it at will, but please, don’t use it in production!

Usage is straightforward, just chmod +x the script, and call it. Use Ctrl-C to exit the script.

January 15, 2015 / mb

Filter out spammers and click bait from Google Analytics

During the last few months, a new wonderful type of spam became part of my life: the Google Analytics spam.

As this article describes, what happens is that you start seeing some blatantly bogus traffic coming from a bunch of websites like semalt.com, buttons-for-website.com, darodar.com, or ilovevitaly.com.

Google announced an Automatic Bot and Spider filtering, but as some users on hacker news reported, it doesn’t work reliably.

So far, the only solution to this problem that worked for me is setting a filter, and add spammers to it as they come. There doesn’t seem to be that many as of today, so this approach is still usable.

[Update – July 2015]: if you have a public HTTP/PHP server available, and are willing to invest half a day to install it, piwik is a nice free, open-source Google Analytics alternative. Piwik uses a community-maintained list of spammers that can also be used in Google Analytics. They wrote a blog post about it, too.

I’ve been using piwik for a few weeks now, and I’m happy with it so far. The nice thing is that updates are very easy to apply, and they include the most recent list of spammers available. The thing that could be improved is the installation process, it’s not as easy as it could be (at least if you’re using Nginx as web server). They also have a Cloud-hosted version, but I guess that if you’re using Google Analytics for free, you’re more interested in free alternatives!

To add a filter in Google Analytics:

  1. go to your Administration page (last tab on your home page)
  2. All filters (on the leftmost column)
  3. New filter
  4. Choose Filter type “Custom” > “Exclude”
  5. Choose “Referral” from the Filter Field menu
  6. Set this as Filter pattern:
    semalt\.com|ilovevitaly\.co|priceg\.com|forum\..*darodar\.com|blackhatworth\.com|hulfingtonpost\.com|buttons-for-website\.com
  7. Select the views that you want to be filtered (I chose “All web site data”)
  8. Save

The filter pattern is a regular expression, so every time you find a new source of spam, simply add another “|spammersite\.com” (remember to escape dots with a backslash, as they mean “any character”).

It’s playing catch-up with spammers, but as long as Google doesn’t find a way to reliably detect them, it’s the only way to get rid of them. I’ve collected those 7 websites in a couple of months, and I’ve seen them being reported by other users as well. Since after setting the filter I’m no longer getting any bogus traffic, it looks like the problem is still relatively small and can be patched on case-by-case basis.

Follow

Get every new post delivered to your Inbox.

Join 95 other followers