Techblog – Page 2 – Articles about technology and software development

Jetty on a vServer that limits numproc and numtcpsock

If you have a vServer it is likely that it has certain restrictions. E.g. many vServers only allow a limited number of threads to be running. You can see these limits using this command:


# cat /proc/user_beancounters

Version: 2.5

       uid  resource                     held              maxheld              barrier                limit              failcnt

 10247230:  kmemsize                 29523869             41385984            106385865            117024451                    0

            lockedpages                     0                    0                  860                  860                    0

            privvmpages                955727              1516817              2097152              2097152                    0

            shmpages                    46762                56592               196608               196608                    0

            dummy                           0                    0  9223372036854775807  9223372036854775807                    0

            numproc                       215                  512                  512                  512                  537

            physpages                 1043148              1048606              1048576              1048576                    0

            vmguarpages                     0                    0              1048576           2147483647                    0

            oomguarpages               601150               808660              1048576           2147483647                    0

            numtcpsock                    274                  581                 1800                 1800                    0

            numflock                        9                   12                  500                  500                    0

            numpty                          5                    6                  128                  128                    0

            numsiginfo                      0                   51                 1024                 1024                    0

            tcpsndbuf                 1129480              3406320              4942675              7056211                    0

            tcprcvbuf                  617208              4396200              4942675              7056211                    0

            othersockbuf                39304               758232               844366              1481926                 2936

            dgramrcvbuf                     0                13344               844366               844366                    0

            numothersock                   54                   77                 1800                 1800                    0

            dcachesize                8079194              8110080              7299072              8110080                    0

            numfile                      1889                 2470                10000                10000                    0

            dummy                           0                    0  9223372036854775807  9223372036854775807                    0

            dummy                           0                    0  9223372036854775807  9223372036854775807                    0

            dummy                           0                    0  9223372036854775807  9223372036854775807                    0

            numiptent                      42                   56                  128                  128                    0

The “numproc” value is the number of allowed threads. In this example the system has exceeded this limit 537 times, which causes problems that e.g. Nagios cannot create sockets or that Java throws OutOfMemory exceptions. Using the program htop you can see how many threads each program uses:

If you are using Jetty or another Java based program you will see a lot of “java” threads. To find out what these threads are actually doing you can use the jstack command using the PID of your main jetty process, e.g. “jstack 12345”. It will then display a stacktrace of all running threads:

# jstack 14843
2019-03-25 10:25:25
Full thread dump Java HotSpot(TM) 64-Bit Server VM:

"Okio Watchdog" #178 daemon prio=5 os_prio=0 tid=0x00007f40a00d2800 nid=0x6de9 in Object.wait() [0x00007f40933e1000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:460)
	at okio.AsyncTimeout.awaitTimeout(AsyncTimeout.java:361)
	at okio.AsyncTimeout$Watchdog.run(AsyncTimeout.java:312)
	- locked <0x00000000e127d148> (a java.lang.Class for okio.AsyncTimeout)

"pool-22-thread-1" #75 prio=5 os_prio=0 tid=0x00007f40942c3000 nid=0x68bf waiting on condition [0x00007f4084900000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000e4860618> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

...

If you have problems that numproc is often exceeded then you can now try to find the threads that shouldn’t be there. E.g. despite I had limited Jettys threadpool it used more and more threads. The reason was that one of my Servlets used a Timer which was only used once but the system didn’t release it after it wasn’t used any more. So more and more Timer threads were created. The solution was to use a ScheduledExecutorService instead as explained on StackOverflow.

Another limitation is often numtcpsock. On my server it is limited to 1800 connections. If that limit is exceeded one cannot connect to the server any more e.g. via ssh. By default jetty does not seem to limit the number of connections it accepts. To limit it you can add these lines to the start.ini file for Jetty 9.4:

jetty.ssl.acceptQueueSize=200
jetty.http.acceptQueueSize=200

--module=connectionlimit
jetty.connectionlimit.maxConnections=200

It limits the number of connections to 200. Without it my server accepted more and more connections without being able to process them in time. After a short time the numtcpsock limit was exceeded and the server was not usable any more. Even my existing ssh connection broke down and I could not connect any more. But with this limit of 200 connections everything seems to work fine.

Fixing Time Machine problems

I had the problem that Time Machine was often stuck in “Preparing backup” or that the backup started and then was interrupted and didn’t finish. If you have similar problems you should check these things: Ensure that there is enough free space on your computer’s hard drive. It seems that Time Machine creates local backups … Continue reading “Fixing Time Machine problems”

Ensure that there is enough free space on your computer’s hard drive. It seems that Time Machine creates local backups on the computer and then uses these local backups to find out what needs to be copied to the backup disk. Additionally it uses snapshots, i.e. it freezes a copy of your hard drive when the backup starts so that the data is consistent. However that also means that files that are deleted during the running backup won’t free any space on your hard drive until the backup is finished. Modified files will exist twice until the backup is finished, i.e. the old and the new version. If these system is running out of space during the backup it seems that it might stop the backup and release the snapshot.
You can use the command

log show --style syslog --predicate "subsystem == "com.apple.TimeMachine" AND processImagePath CONTAINS "backupd"" --info

to see what the Time Machine is doing and check if any errors are displayed.
You can use the command
sudo sysctl debug.lowpri_throttle_enabled=0
to increase the speed of the backup by making it to continue to run the backup even while you are using your computer. Otherwise if the computer is too busy doing other things the backup will be paused until the computer is idle again.

Especially after freeing about 7% (70GB of 1TB) of space the backup seems to work fine now. I hope this will help you.

Android: Notifications about changes in shared calendars

If you are sharing a calendar with another person or your family you probably want to know when events are added, deleted or modified. Because the calendar is shared you will see these new and modified events automatically in your calendar. However you will often see these events only a few days in advance because … Continue reading “Android: Notifications about changes in shared calendars”

If you are sharing a calendar with another person or your family you probably want to know when events are added, deleted or modified. Because the calendar is shared you will see these new and modified events automatically in your calendar. However you will often see these events only a few days in advance because you don’t watch the next weeks of your calendar all the time. And if an event has been deleted you won’t see it at all and will have to remember that there was an event.

An easy way to see which events have changed is the app “Calendar Watcher” . Just select the calendars you would like to watch and you will get status bar notifications about all changes in these calendars:

If an event was moved to another time, it will even show you the old and the new time of the event.

Signing a JavaFX Maven project for MacOS’ Guardkeeper

When creating a JavaFX application for MacOS as an “.app” file it is important to sign it correctly. If it isn’t signed correctly, the message “YourApp can’t be opened because it is from an unidentified developer.” will appear when a user tries to start the downloaded “.app” file: To fix this one can use the … Continue reading “Signing a JavaFX Maven project for MacOS’ Guardkeeper”

To fix this one can use the javafx-maven-plugin. When using it you just have to add some information about your configuration to your pom.xml file. You can find further information on the project’s website but here is an example:

<plugin>
    <groupId>com.zenjava</groupId>
    <artifactId>javafx-maven-plugin</artifactId>
    <version>8.7.0</version>
    <configuration>
        <vendor>YourCompany</vendor>
        <updateExistingJar>true</updateExistingJar>
        <mainClass>YourMainClass</mainClass>
        <appName>YourAppName</appName>
        <verbose>true</verbose>
        <jvmArgs>
            <argument>-Djava.library.path=. -Xms168m -Xmx1024m -XX:+UseG1GC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=30</argument>
        </jvmArgs>
        <bundleArguments>
            <mac.signing-key-developer-id-app>ABC</mac.signing-key-developer-id-app>
            <mac.signing-key-app>ABC</mac.signing-key-app>
            <mac.signing-key-developer-id-installer>DEF</mac.signing-key-developer-id-installer>
            <mac.signing-key-pkg>GHI</mac.signing-key-pkg>
        </bundleArguments>
        <additionalAppResources>libs</additionalAppResources>
    </configuration>
    <executions>
        <execution>
            <id>create-jfxjar</id>
            <phase>package</phase>
            <goals>
                <goal>build-jar</goal>
            </goals>
        </execution>
    </executions>
</plugin>

The important part is in the bundleArguments section. There you can define some parameters for the javapackager, which creates the native application. Depending on if you would like to distribute your application directly or via the Mac AppStore you have to enter different keys here. You can either enter the key’s name or its SHA-1 number (without spaces). Both can be found in the “Keychain Access” program.

After you have finished creating your “.app” file you can check if Guardkeeper will accept it:

# spctl -a -vvvv YourApplication.app

YourApplication.app: accepted
source=Developer ID
origin=Developer ID Application: YourName (YourID)

If it says “accepted” then it should work.

Creating and Testing an ANTLR parser with Intellij IDEA or Android Studio

In this example we will parse a simple text with ANTLR to see how to set up and use ANTLR with Intellij IDEA or Android Studio. Afterward you can extend the example or write your own files to parse more complex input. First you need to install the ANTLR plugin. Open the preferences and select … Continue reading “Creating and Testing an ANTLR parser with Intellij IDEA or Android Studio”

First you need to install the ANTLR plugin. Open the preferences and select “Plugins > Browse repositories”:

Then select the ANTLR plugin:

Now create an empty file and name it “Test.g4”. Write the following lines into the file:

grammar Test;
main: 'Hello ' name '!';
name: ANY+;
ANY: .;

Then click on ANTLR Preview:

A new pane should appear that should display “text.g4 start rule: <select from navigator or grammar>”. To fix this, right click on “main” in the “Test.g4” file and select “Test Rule main”:

Now type “Hello John!” into the text area of the ANTLR Preview pane. On the right side you should see the parsed result:

As you can see, it has correctly parsed the name into a separate tag. If you enter an invalid text it will show an error.

Now let’s see how we can access the data from Java/Kotlin. Right click your “Test.g4” file and select “Configure ANTLR”.

Enter the root path where your project’s sources are saved and the package name that the parser should use. And specify “Java” as language:

Then click on “Generate ANTLR Recognizer”:

Now you could write the following code in Kotlin

val lexer = TestLexer(CharStreams.fromString("Hallo John!"))
val parser = TestParser(CommonTokenStream(lexer))

val name = parser.main().name().text

or in Java

TestLexer lexer = new TestLexer(CharStreams.fromString("Hallo John!"));
TestParser parser = new TestParser(new CommonTokenStream(lexer));

String name = parser.main().name().getText();

and the variable “name” would contain the parsed name “John”.

To use this code you also need the ANTLR Runtime in your project. If you are using Gradle e.g. with Android Studio you have to add this line to the “dependencies” section of your app’s build.gradle:

implementation 'org.antlr:antlr4-runtime:4.7'

If you are using Maven in your project you have to add this to your “dependencies” section:


    org.antlr
    antlr4-runtime
    4.7

Too see which rules you can use in the g4 file you can use the documentation here:

Lexer Rules
Parser rules

Using Antlr to parse date ranges in Java and Kotlin

I wanted to parse date ranges that could occur as e.g. “01.01.” or “01.01.-05.01.” or “01.01.-05.01./09.01.” or similar combinations. To make it easier to correctly parse all possible combinations I have used Antlr to parse the dates. First I had to create rules in a file that I named “Dates.g4” that define what is a … Continue reading “Using Antlr to parse date ranges in Java and Kotlin”

First I had to create rules in a file that I named “Dates.g4” that define what is a valid date range:

grammar Dates;
r: (element (divider? element)*);
element: (daterange | singledate);
daterange: date minus date;
singledate: date;
minus: '-' | '–';
divider: '/';
date: day '.' month ('.')?;
day: INT;
month: INT;
INT: [0-9]+;
WS: [ \t\r\n]+ -> skip ;

Let’s see what this does. The “grammar” line just defines a name. The next line defines a token “r” that can consist of an “element” and an arbitrary number of “divider” objects (or no divider) and another element. The next line defines what such an “element” is. It is either a “daterange” or a “singledate”. And so on, all tokens are defined this way. A question mark makes the element optional, i.e. it does not need to be in the parsed text.

The rules in uppercase letters are lexer rules, i.e. they don’t use self defined tokens to define the structure of the parsed text but they define characters that should be allowed.

I have used the Intellij IDEA IDE with the Antlr plugin. So to generate the necessary Java classes from the *.g4 file above I just had to right click the *.g4 file and choose “Generate ANTLR Recognizer”:

This creates several classes in the directory and package that you can change by clicking on “Configure ANTLR” in the menu above.

The generated classes are easy to use, e.g. in Kotlin:

val lexer = DatesLexer(CharStreams.fromString(text))
val parser = DatesParser(CommonTokenStream(lexer))

val parsed = parser.r()
for (element in parsed.element()) {

Inside the loop you can now access the dates e.g. with

element.daterange()

and

element.singledate()

because as defined in the *.g4 file above an element contains either a “daterange” or a “singledate”. As you can see the generated functions use the names that were specified in the *.g4 file.

Automatic uploads of Android apk files to Google Play

To save time you can automate the process of uploading new apk files to Google Play using tools like gradle-play-publisher or fastlane. The process is simple. First you have to create a Google Service account and download the created p12 file. In the Google Play console you have to configure the access for this account. … Continue reading “Automatic uploads of Android apk files to Google Play”

To save time you can automate the process of uploading new apk files to Google Play using tools like gradle-play-publisher or fastlane.

The process is simple. First you have to create a Google Service account and download the created p12 file. In the Google Play console you have to configure the access for this account. And you have to add a few sections to your project’s gradle files. You can find instructions for it on the gradle-play-publisher website.

Then you can download all your existing descriptions with this command:

./gradlew bootstrapReleasePlayResources

And to just upload a new apk file for a release or a alpha/beta version use this command:

./gradlew clean publishApkRelease

or this if you are using App Bundles

./gradlew clean publishBundle

It also uploads the contents of the “whatsnew” files as changelog to the Google Play store.

Parsing PDF files with Java or Kotlin

Often information is not available in a computer readable format like JSON, XML or CSV. When only a human readable PDF file is available, one can try to use a PDF parser to retrieve the needed information. There is e.g. Apache Tika that can read PDF files and return the contents as tokens. It can … Continue reading “Parsing PDF files with Java or Kotlin”

Often information is not available in a computer readable format like JSON, XML or CSV. When only a human readable PDF file is available, one can try to use a PDF parser to retrieve the needed information. There is e.g. Apache Tika that can read PDF files and return the contents as tokens. It can be quite useful but it doesn’t return tabular information so if you have a table with empty cells you don’t see which cells are empty and it can be difficult to know to which cell the returned data belongs to.

For this purpose another library called Tabula exists. It provides an easy to use local web page that allows to the tables of a PDF file and export them as CSV or JSON files:

You can also embed tabula-java into an own program to use it e.g. in batch jobs. E.g. this Kotlin snippet loads a PDF file pdfFile and writes its contents as JSON into tmpfile:

val tmpfile = File.createTempFile("pdfparser", "json")
val args = arrayOf(pdfFile.absolutePath, "-g", "-l", "-f", "JSON", "-o", tmpfile.absolutePath)

val parser = DefaultParser()
val cmd = parser.parse(CommandLineApp.buildOptions(), args)

val stringBuilder = StringBuilder()
CommandLineApp(stringBuilder, cmd).extractTables(cmd)

From there you can parse the JSON to process it further.

Syncing two calendars (e.g. Exchange into Google Calendar) on iOS and Android

Sometimes people are allowed to see their work calendar on their Android phone or iPhone. But they cannot share the calendar directly with their family to show them when they have work events. For this purpose it would be useful to mirror the work calendar (e.g. the Exchange calendar) into a Google calendar in their … Continue reading “Syncing two calendars (e.g. Exchange into Google Calendar) on iOS and Android”

Sometimes people are allowed to see their work calendar on their Android phone or iPhone. But they cannot share the calendar directly with their family to show them when they have work events. For this purpose it would be useful to mirror the work calendar (e.g. the Exchange calendar) into a Google calendar in their Google account so that they can share it with their family. By mirroring only the times of the events but not the contents (title, location, description) confidential data can be removed and only the important data (the time of the events) is shared.

If you have that problem and would like your family to see your work Exchange calendar, then just do the following:

Create a new empty calendar e.g. “Work” in your Google Calendar account.
Sync your Android phone/iPhone with your Exchange account and your Google Calendar account so that you can see calendars from both accounts.
Install SyncCal for Android ( https://play.google.com/store/apps/details?id=com.calengoo.synccal ) or SyncCal for iOS ( https://itunes.apple.com/us/app/synccal/id796482010?mt=8 ) on your phone and use it to copy the Exchange calendar into your new “Work” calendar in your Google account.
Check if you can see your work events in Google Calendar. If that works fine, you can share the calendar with your family to let them see your work events.

Checking remote webspace with Nagios

My webspace provider offers a simple ssh access to upload and download files. One cannot install any software there and has a limited set of commands. I wanted to check the free space on that server because it often runs out of free space and then new uploads fail. The script on the server The … Continue reading “Checking remote webspace with Nagios”

The script on the server

The server does not have a “quota” command and “df” is about the whole filesystem, not just about my webspace. So I used “du -ms .” to check how much data is saved in my webspace. I have creates a small shell script “checkfreespace.sh” and saved it on the server:

#!/bin/bash
export USEDSPACE=`du -ms . | sed "s/[^0-9]*//g"`;
if [ "$USEDSPACE" -lt "9000" ] ; then echo "OK - $USEDSPACE MB used" ; exit 0; else echo "CRITICAL - $USEDSPACE MB used"; exit 2; fi

It checks if less than 9000MB are used and returns “OK” with exit code 0. Otherwise it returns “CRITICAL” with exit code 2. These are keywords that are used by Nagios.

SSH configuration

Then it is necessary to make it possible to login using public key authentication (so that Nagios does not need a password).

Use “chsh nagios” to set the shell of nagios to “/bin/bash”.
Use “su – nagios” to switch to the home directory of nagios.
Create a key pair if necessary e.g. using
```
ssh-keygen -t rsa -b 4096 -C your@email.com
```
in the .ssh directory of the nagios user and change the owner to the nagios user if necessary.
Append the contents of the file .ssh/id_rsa.pub to the file .ssh/authorized_keys on the server.
Use “chsh nagios” to set the shell of nagios back to “/bin/false”.

Nagios configuration

Finally you have to configure Nagios. Add this to the commands.cfg file:

define command{
	command_name	check_my_free_space
	command_line	/usr/lib/nagios/plugins/check_by_ssh -H ssh.yourserver.com -l yourusername -C "bash checkfreespace.sh" -t 30
	}

Afterward you have to use this new command in your services file, e.g. add this:

define service {
        service_description             Free webspace;
        check_command                   check_my_free_space
        use                             generic-service;
        host_name  android.calengoo.com;
        notification_interval           0 ; set > 0 if you want to be renotified
        normal_check_interval           60
}