IntelliJ Idea adds tags when typing ‘t

This was quite annoying, especially since I failed at googling a cure for it.

What happens: whenever you type “'t” in a comment on a Java file in Idea followed by a whitespace, it gets replaced with “<></>“, and whatever you type after that is added as if it were an HTML tag pair.

If you actually pause after typing “'t” (which I don’t know why it’s something I never did until now), you’ll notice IntelliJ’s auto-complete popup telling you what’s going on:

popup

It turns out that “t” is the shortcut for “tag pair”, which might be awesome for HTML coding, but it’s not as nice when you’re writing comments.

To disable that, go to Settings/Editor/Live Templates/, expand the html/xml line, and uncheck t (tag pair). Hit OK, and it’s gone!

Advertisements

After upgrading, Jenkins doesn’t start: org.xml.sax.saxparseexception: premature end of file

I recently upgraded my Jenkins installation using Debian’s package manager, and to my surprise… Jenkins wouldn’t start!

The error logged in /var/log/jenkins/jenkins.log looked like this:

WARNING: Failed startup of context w.{,file:/var/cache/jenkins/war/},/var/cache/jenkins/war
org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.

followed by a Java stack trace. I noticed a suspicious line in that stack trace:

at org.eclipse.jetty.webapp.MetaData.setWebXml(MetaData.java:170)

interesting. It looked like something was wrong with Jenkins’ web.xml, so I checked /var/cache/jenkins/war/WEB-INF/web.xml and, surprise surprise… it was empty!

I unpacked an old jenkins.war that I had, and copied its web.xml over, restarting the service. A-ha! The logged error changed. Unfortunately, it still wouldn’t start, but it made me look for “jenkins corrupted /var/cache” which finally lead me to this bug report.

“Dang!” – I thought – “is that it?”.

Moved the old /var/cache/jenkins away, created a new empty directory, and set user jenkins as its owner:

mv /var/cache/jenkins ~/var-cache-jenkins-back
mkdir /var/cache/jenkins
chown -R jenkins:jenkins /var/cache/jenkins
service jenkins start

sonofa… it worked!

A template to create log4j Logger objects in Eclipse

In my (server) Java apps, I usually use log4j to keep logs. These days, I actually use SLF4J as a proxy for log4j, mostly because the framework I’m using (Vert.x) uses it.

Over the years I’ve tuned my log format to only store the information I care about, and nothing more than that:

<PatternLayout pattern="[%d{HH:mm:ss.SSS}] %p %c{3} - %m%n" />

so I have a timestamp, the log level (%p), the class name with up to 3 levels of package hierarchy (%c{3}), and the new-line-terminated message (%m%n).

A sample line:

[11:39:36.667] TRACE redis.redis_client.RedisDecoder - received new MultiBulkReply

This format requires a Logger object in all classes that need logging, which requires quite a bit of boilerplate code, e.g.:

private static final Logger LOGGER = LoggerFactory
    .getLogger(SlackRouter.class);

You quickly grow tired of typing all of that. But worry not! Eclipse comes to the rescue!

Setup an Eclipse Template

I created a template that does the work for me: now, all I have to do to add a new Logger object is type log, hit Ctrl + space, and select “Insert logger”.

Just like this:

Adding a Logger

Note that typing “log4” instead of “log” only gives you one option, saving 2 precious key strokes! 😛

Here’s how to set it up for your Eclipse.

  1. Open your Eclipse Preferences menu
  2. Type “java editor templates” in the search bar
    uno
  3. Hit the “New” button
  4. Set “log4j” as name (or whatever you want the shortcut to be), add a description, paste this in the “Pattern” field:
    ${:import(org.slf4j.Logger,org.slf4j.LoggerFactory)}
    private static final Logger LOGGER = LoggerFactory.getLogger(${enclosing_type}.class);
    

    and hit “OK”
    tre

  5. That’s it!

    If you’re not using SLF4J, all you need to do is change the import code to include the actual class you use.

Find the most recent file/folder in a folder with Java 8

A simple snippet that shows how Java 8 lambdas can be really nice to replace operations that used to require a lot of boilerplate in earlier versions of Java.

If you want to find the most recent file or subfolder in a folder with Java 8, here’s what you need to do:

Path parentFolder = Paths.get("path", "to", "your", "file");

Optional<File> mostRecentFileOrFolder =
    Arrays
        .stream(parentFolder.toFile().listFiles())
        .max(
            (f1, f2) -> Long.compare(f1.lastModified(),
                f2.lastModified()));

if (mostRecentFolder.isPresent()) {
    File mostRecent = mostRecentFileOrFolder.get();
    System.out.println("most recent is " + mostRecent.getPath());
} else {
    System.out.println("folder is empty!");
}

The very nice thing is that you can take advantage of the flexibility of Java 8 streams to either make the operation parallel (just throw in a parallel() call after Arrays.stream()), or to filter results according to other criteria.

For example, if you’re only interested in one type of child elements (a file or a folder), you could…

// if you're only interested in files...
Optional<File> mostRecentFile =
    Arrays
        .stream(parentFolder.toFile().listFiles())
        .filter(f -> f.isFile())
        .max(
            (f1, f2) -> Long.compare(f1.lastModified(),
                f2.lastModified()));

// if you're interested in folders...
Optional<File> mostRecentFolder =
    Arrays
        .stream(parentFolder.toFile().listFiles())
        .filter(f -> f.isDirectory())
        .max(
            (f1, f2) -> Long.compare(f1.lastModified(),
                f2.lastModified()));

ForkedBooter steals window focus on Mac OS while Maven is running

This is about running Maven, specifically.

For most Java applications, any of the solutions from this question on StackOverflow should work.

It’s a matter of passing -Djava.awt.headless=true to the Java process, which can be done for all Java processes by saving that option to a variable named JAVA_TOOL_OPTIONS in your .bashrc.

That’s fine as long as you have direct control over the Java process, which is not the case with Maven.

In my case, it was the Failsafe Maven Plugin that caused that annoyance, but some were experiencing the same behavior with an old version of Surefire. The following snippet fixes both.

In your pom.xml, change your existing <plugin> definitions so that they look like this (the important lines are highlighted):

<!-- this is inside your <project><build><plugins> block -->
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>${maven.surefire.plugin.version}</version>
    <configuration>
      <includes>
        <include>**/unit/**/*Test*.java</include>
      </includes>
      <!-- prevent the annoying ForkedBooter process from stealing 
        window focus on Mac OS -->
      <argLine>-Djava.awt.headless=true</argLine>
    </configuration>
  </plugin>
  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>${maven.failsafe.plugin.version}</version>
    <configuration>
      <!-- prevent the annoying ForkedBooter process from stealing 
        window focus on Mac OS -->
      <argLine>-Djava.awt.headless=true</argLine>
      <includes>
        <include>**/integration/**/*Test*</include>
      </includes>
    </configuration>
    <executions>
      <execution>
        <goals>
          <goal>integration-test</goal>
          <goal>verify</goal>
        </goals>
      </execution>
    </executions>
  </plugin>

Done!

I added this reply to stack exchange as well, but maybe this post will be easier to find (as that question already already had an accepted answer when I replied, and it doesn’t work for Maven). We’ll see!

Run JUnit tests in order

JUnit runs test methods in whatever order it wants, which is generally fine as long as all your test methods are independent of each other. This, though, may not be the case when you’re writing integration tests, where you could have a sequence of operations like

login(credentials);
requestProject("foo");
createNewItem("bar");
logout();

in a situation like this, you always want testLogout() to run after testLogin(), and createNewItem() to run after requestProject()!

Yes, you could group everything in a single test method, but it may become huge and very hard to maintain.

Quite surprisingly, JUnit doesn’t have a built-in solution for this. You can run test methods sorted by name using the @FixMethodOrder(MethodSorters.NAME_ASCENDING) tag, but then you need to artificially name your methods so that they appear in the order you want.

I found some alternative solutions in this stackoverflow question, but they require annotating your methods with tags to specify the order in which you want them to run. What I’d like them to do is just run in the same order as they appear in the source code for the test class. Among the answers, I just found this blog post that achieves the same result as I did, only it looks somewhat more complicated (it involves writing/including several classes).

My solution is fairly simple, but there’s 2 warnings:

  1. it uses Javassist, so if you don’t want to add libraries, there’s that
  2. it only works as long as you don’t have test classes that extend other test classes, and you don’t override all @Test-annotated methods in the subclass (I’ve never done that, but I guess as tests get complicated, you may have that); this can be fixed quite easily though, you just need to add the logic for what should come first according to your needs

On with the code!

You can grab the source straight from this pastebin, or copy/paste it from here (I added the MIT License, I think it should be the most permissive.. my intent is to say “do whatever the heck you want with this code”)

/*
 * Copyright (C) <2014> <Michele Bonazza>
 * 
 * 
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 * 
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 * 
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 * SOFTWARE.
 */

/**
 * A test runner that runs tests according to their position in the source file
 * of the test class.
 * 
 * @author Michele Bonazza
 */
public class OrderedTestRunner extends BlockJUnit4ClassRunner {

    /**
     * Creates a new runner
     * 
     * @param clazz
     *            the class being tested
     * @throws InitializationError
     *             if something goes wrong
     */
    public OrderedTestRunner(Class<?> clazz) throws InitializationError {
        super(clazz);
    }

    /*
     * (non-Javadoc)
     * 
     * @see org.junit.runners.BlockJUnit4ClassRunner#computeTestMethods()
     */
    @Override
    protected List<FrameworkMethod> computeTestMethods() {
        // get all methods to be tested
        List<FrameworkMethod> toSort = super.computeTestMethods();

        if (toSort.isEmpty())
            return toSort;

        // a map containing <line_number, method>
        final Map<Integer, FrameworkMethod> testMethods = new TreeMap<>();

        // check that all methods here are declared in the same class, we don't
        // deal with test methods from superclasses that haven't been overridden
        Class<?> clazz = getDeclaringClass(toSort);
        if (clazz == null) {
            // fail explicitly
            System.err
                    .println("OrderedTestRunner can only run test classes that"
                            + " don't have test methods inherited from superclasses");
            return Collections.emptyList();
        }

        // use Javassist to figure out line numbers for methods
        ClassPool pool = ClassPool.getDefault();
        try {
            CtClass cc = pool.get(clazz.getName());
            // all methods in toSort are declared in the same class, we checked
            for (FrameworkMethod m : toSort) {
                String methodName = m.getName();
                CtMethod method = cc.getDeclaredMethod(methodName);
                testMethods.put(method.getMethodInfo().getLineNumber(0), m);
            }
        } catch (NotFoundException e) {
            e.printStackTrace();
        }

        return new ArrayList<>(testMethods.values());
    }

    private Class<?> getDeclaringClass(List<FrameworkMethod> methods) {
        // methods can't be empty, it's been checked
        Class<?> clazz = methods.get(0).getMethod().getDeclaringClass();

        for (int i = 1; i < methods.size(); i++) {
            if (!methods.get(i).getMethod().getDeclaringClass().equals(clazz)) {
                // they must be all in the same class
                return null;
            }
        }

        return clazz;
    }
}

to use this, you need to add Javassist to your classpath; if you have a Maven project, it’s incredibly easy to do so, just add this to your POM:

<dependency>
  <groupId>javassist</groupId>
  <artifactId>javassist</artifactId>
  <version>3.12.1.GA</version>
</dependency>

and annotate your JUnit test class with @RunWith(OrderedTestRunner.class), like this:

import wherever.you.put.OrderedTestRunner;
import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(OrderedTestRunner.class)
public class MyTestClass {

    @Test
    public void testZero() {
        System.out.println("test zero run");
    }

    @Test
    public void testOne() {
        System.out.println("test one run");
    }
}

That’s it!

About warning 2. above, in case you have something like MyTestClass extends BaseTestClass and you have methods annotated with @Test in BaseTestClass that aren’t overridden by MyTestClass, OrderedTestRunner will just fail printing the message you can see at line 50 above on System.err. I did this because I don’t think there’s a well-defined order in that case (should all methods from the super class run first? Should that go all the way up in the class hierarchy?), so you can adjust it to fit your particular needs.

HttpPost requests executed multiple times (Apache HttpClient)

This is something I noticed on Android, but from what I read it also involves the desktop Java version.

I was sending POST requests to an API server, and I was getting some random 400 Bad Request responses from time to time. I wish Apache provided an easy way to log the plain text version of Http requests, but I couldn’t find a better way to see what the app was sending than sending the same request to my PC when failing.

So to log requests I start netcat (sudo nc -l 80 on a mac) or a very minimal server in python (it’s more or less the same as the example on Twisted’s front page) and route them there whenever an error occurs.

try {
   response = client.execute(post,
                  new BasicResponseHandler());
} catch (IOException e) {
   if (DEBUG_FAILED_REQUESTS) {
      post.setURI(URI.create(DEBUG_FAILED_REQUESTS_SERVER));
      try {
         client.execute(post, new BasicResponseHandler());
      } catch (IOException e1) {
         e1.printStackTrace();
      }
   }
}

I don’t know if it’s my router, but sometimes connections from the Android device to my PC get blocked: to make them work I just open a browser on the Android, go to some website and then try again with my internal IP (192.168.0.whatever). It always works, no idea why.

Using this code I discovered that my post requests were executed 4 times each, nearly at the same time. I discovered that it’s the default behavior, and you must provide your own RetryHandler if you want the HttpClient to work otherwise.

In my case, my calls are sent to Google’s shortener service, and for some reason sometimes it just rejects requests. If you wait a little bit between attempts you increase your chance of getting valid responses. So this is what I did:

HttpPost post = new HttpPost(SHORTENER_URL);
String shortURL = null;
int tries = 0;
try {
    post.setEntity(new StringEntity(String.format(
            "{\"longUrl\": \"%s\"}",
            getURL(encodedID, encodedAssignedID))));
    post.setHeader("Content-Type", "application/json");
    DefaultHttpClient client = new DefaultHttpClient();
    // disable default behavior of retrying 4 times in a burst
    client.setHttpRequestRetryHandler(new DefaultHttpRequestRetryHandler(
            0, false));
    String response = null;
    while (response == null && tries < RETRY_COUNT) {
        try {
            response = client.execute(post,
                    new BasicResponseHandler());
        } catch (IOException e) {
            // maybe just try again...
            tries++;
            Utils.debug("attempt %d failed... waiting", tries);
            try {
                // life is too short for exponential backoff
                Thread.sleep(RETRY_SLEEP_TIME * tries);
            } catch (InterruptedException e1) {
                e1.printStackTrace();
            }
        }
    }
    Utils.debug("response is %s", response);
    if (response != null) {
        JSONObject jsonResponse = new JSONObject(response);
        shortURL = jsonResponse.getString("id");
    } else if (DEBUG_FAILED_REQUESTS) {
        Utils.debug("attempt %d failed, giving up", RETRY_COUNT);
        debugPost(post, client);
    }
} catch (JSONException e) {
    e.printStackTrace();
} catch (UnsupportedEncodingException e) {
    e.printStackTrace();
}

where debugPost() is a method that calls my PC to log the request, and Utils.debug() is just a small utility method I wrote to log messages with logcat using String.format() if format args are passed to it (it also takes care of splitting messages that would be truncated by logcat itself).

You could choose to implement exponential backoff very easily, but since it’s a blocking operation for the user in my case I preferred not to.