Creating and Testing an ANTLR parser with Intellij IDEA or Android Studio

In this example we will parse a simple text with ANTLR to see how to set up and use ANTLR with Intellij IDEA or Android Studio. Afterward you can extend the example or write your own files to parse more complex input. First you need to install the ANTLR plugin. Open the preferences and select … Continue reading “Creating and Testing an ANTLR parser with Intellij IDEA or Android Studio”

In this example we will parse a simple text with ANTLR to see how to set up and use ANTLR with Intellij IDEA or Android Studio. Afterward you can extend the example or write your own files to parse more complex input.

First you need to install the ANTLR plugin. Open the preferences and select “Plugins > Browse repositories”:

Then select the ANTLR plugin:

Now create an empty file and name it “Test.g4”. Write the following lines into the file:

grammar Test;
main: 'Hello ' name '!';
name: ANY+;
ANY: .;

Then click on ANTLR Preview:

A new pane should appear that should display “text.g4 start rule: <select from navigator or grammar>”. To fix this, right click on “main” in the “Test.g4” file and select “Test Rule main”:

Now type “Hello John!” into the text area of the ANTLR Preview pane. On the right side you should see the parsed result:

As you can see, it has correctly parsed the name into a separate tag. If you enter an invalid text it will show an error.

Now let’s see how we can access the data from Java/Kotlin. Right click your “Test.g4” file and select “Configure ANTLR”.

Enter the root path where your project’s sources are saved and the package name that the parser should use. And specify “Java” as language:

Then click on “Generate ANTLR Recognizer”:

Now you could write the following code in Kotlin

val lexer = TestLexer(CharStreams.fromString("Hallo John!"))
val parser = TestParser(CommonTokenStream(lexer))

val name = parser.main().name().text

or in Java

TestLexer lexer = new TestLexer(CharStreams.fromString("Hallo John!"));
TestParser parser = new TestParser(new CommonTokenStream(lexer));

String name = parser.main().name().getText();

and the variable “name” would contain the parsed name “John”.

To use this code you also need the ANTLR Runtime in your project. If you are using Gradle e.g. with Android Studio you have to add this line to the “dependencies” section of your app’s build.gradle:

implementation 'org.antlr:antlr4-runtime:4.7'

If you are using Maven in your project you have to add this to your “dependencies” section:


    org.antlr
    antlr4-runtime
    4.7

Too see which rules you can use in the g4 file you can use the documentation here:

Lexer Rules
Parser rules

Using Antlr to parse date ranges in Java and Kotlin

I wanted to parse date ranges that could occur as e.g. “01.01.” or “01.01.-05.01.” or “01.01.-05.01./09.01.” or similar combinations. To make it easier to correctly parse all possible combinations I have used Antlr to parse the dates. First I had to create rules in a file that I named “Dates.g4” that define what is a … Continue reading “Using Antlr to parse date ranges in Java and Kotlin”

I wanted to parse date ranges that could occur as e.g. “01.01.” or “01.01.-05.01.” or “01.01.-05.01./09.01.” or similar combinations. To make it easier to correctly parse all possible combinations I have used Antlr to parse the dates.

First I had to create rules in a file that I named “Dates.g4” that define what is a valid date range:

grammar Dates;
r: (element (divider? element)*);
element: (daterange | singledate);
daterange: date minus date;
singledate: date;
minus: '-' | '–';
divider: '/';
date: day '.' month ('.')?;
day: INT;
month: INT;
INT: [0-9]+;
WS: [ \t\r\n]+ -> skip ;

Let’s see what this does. The “grammar” line just defines a name. The next line defines a token “r” that can consist of an “element” and an arbitrary number of “divider” objects (or no divider) and another element. The next line defines what such an “element” is. It is either a “daterange” or a “singledate”. And so on, all tokens are defined this way. A question mark makes the element optional, i.e. it does not need to be in the parsed text.

The rules in uppercase letters are lexer rules, i.e. they don’t use self defined tokens to define the structure of the parsed text but they define characters that should be allowed.

I have used the Intellij IDEA IDE with the Antlr plugin. So to generate the necessary Java classes from the *.g4 file above I just had to right click the *.g4 file and choose “Generate ANTLR Recognizer”:

This creates several classes in the directory and package that you can change by clicking on “Configure ANTLR” in the menu above.

The generated classes are easy to use, e.g. in Kotlin:

val lexer = DatesLexer(CharStreams.fromString(text))
val parser = DatesParser(CommonTokenStream(lexer))

val parsed = parser.r()
for (element in parsed.element()) {

Inside the loop you can now access the dates e.g. with

element.daterange()

and

element.singledate()

because as defined in the *.g4 file above an element contains either a “daterange” or a “singledate”. As you can see the generated functions use the names that were specified in the *.g4 file.

Automatic uploads of Android apk files to Google Play

To save time you can automate the process of uploading new apk files to Google Play using tools like gradle-play-publisher or fastlane. The process is simple. First you have to create a Google Service account and download the created p12 file. In the Google Play console you have to configure the access for this account. … Continue reading “Automatic uploads of Android apk files to Google Play”

To save time you can automate the process of uploading new apk files to Google Play using tools like gradle-play-publisher or fastlane.

The process is simple. First you have to create a Google Service account and download the created p12 file. In the Google Play console you have to configure the access for this account. And you have to add a few sections to your project’s gradle files. You can find instructions for it on the gradle-play-publisher website.

Then you can download all your existing descriptions with this command:

./gradlew bootstrapReleasePlayResources

And to just upload a new apk file for a release or a alpha/beta version use this command:

./gradlew clean publishApkRelease

or this if you are using App Bundles

./gradlew clean publishBundle

It also uploads the contents of the “whatsnew” files as changelog to the Google Play store.

Parsing PDF files with Java or Kotlin

Often information is not available in a computer readable format like JSON, XML or CSV. When only a human readable PDF file is available, one can try to use a PDF parser to retrieve the needed information. There is e.g. Apache Tika that can read PDF files and return the contents as tokens. It can … Continue reading “Parsing PDF files with Java or Kotlin”

Often information is not available in a computer readable format like JSON, XML or CSV. When only a human readable PDF file is available, one can try to use a PDF parser to retrieve the needed information. There is e.g. Apache Tika that can read PDF files and return the contents as tokens. It can be quite useful but it doesn’t return tabular information so if you have a table with empty cells you don’t see which cells are empty and it can be difficult to know to which cell the returned data belongs to.

For this purpose another library called Tabula exists. It provides an easy to use local web page that allows to the tables of a PDF file and export them as CSV or JSON files:

Tabula screenshot running on localhost

You can also embed tabula-java into an own program to use it e.g. in batch jobs. E.g. this Kotlin snippet loads a PDF file pdfFile and writes its contents as JSON into tmpfile:

val tmpfile = File.createTempFile("pdfparser", "json")
val args = arrayOf(pdfFile.absolutePath, "-g", "-l", "-f", "JSON", "-o", tmpfile.absolutePath)

val parser = DefaultParser()
val cmd = parser.parse(CommandLineApp.buildOptions(), args)

val stringBuilder = StringBuilder()
CommandLineApp(stringBuilder, cmd).extractTables(cmd)

From there you can parse the JSON to process it further.

Syncing two calendars (e.g. Exchange into Google Calendar) on iOS and Android

Sometimes people are allowed to see their work calendar on their Android phone or iPhone. But they cannot share the calendar directly with their family to show them when they have work events. For this purpose it would be useful to mirror the work calendar (e.g. the Exchange calendar) into a Google calendar in their … Continue reading “Syncing two calendars (e.g. Exchange into Google Calendar) on iOS and Android”

Sometimes people are allowed to see their work calendar on their Android phone or iPhone. But they cannot share the calendar directly with their family to show them when they have work events. For this purpose it would be useful to mirror the work calendar (e.g. the Exchange calendar) into a Google calendar in their Google account so that they can share it with their family. By mirroring only the times of the events but not the contents (title, location, description) confidential data can be removed and only the important data (the time of the events) is shared.

If you have that problem and would like your family to see your work Exchange calendar, then just do the following:

  1. Create a new empty calendar e.g. “Work” in your Google Calendar account.
  2. Sync your Android phone/iPhone with your Exchange account and your Google Calendar account so that you can see calendars from both accounts.
  3. Install SyncCal for Android ( https://play.google.com/store/apps/details?id=com.calengoo.synccal ) or SyncCal for iOS ( https://itunes.apple.com/us/app/synccal/id796482010?mt=8 ) on your phone and use it to copy the Exchange calendar into your new “Work” calendar in your Google account.
  4. Check if you can see your work events in Google Calendar. If that works fine, you can share the calendar with your family to let them see your work events.

Apple AppStore Available on Google Play

Checking remote webspace with Nagios

My webspace provider offers a simple ssh access to upload and download files. One cannot install any software there and has a limited set of commands. I wanted to check the free space on that server because it often runs out of free space and then new uploads fail. The script on the server The … Continue reading “Checking remote webspace with Nagios”

My webspace provider offers a simple ssh access to upload and download files. One cannot install any software there and has a limited set of commands. I wanted to check the free space on that server because it often runs out of free space and then new uploads fail.

The script on the server

The server does not have a “quota” command and “df” is about the whole filesystem, not just about my webspace. So I used “du -ms .” to check how much data is saved in my webspace. I have creates a small shell script “checkfreespace.sh” and saved it on the server:

#!/bin/bash
export USEDSPACE=`du -ms . | sed "s/[^0-9]*//g"`;
if [ "$USEDSPACE" -lt "9000" ] ; then echo "OK - $USEDSPACE MB used" ; exit 0; else echo "CRITICAL - $USEDSPACE MB used"; exit 2; fi

It checks if less than 9000MB are used and returns “OK” with exit code 0. Otherwise it returns “CRITICAL” with exit code 2. These are keywords that are used by Nagios.

SSH configuration

Then it is necessary to make it possible to login using public key authentication (so that Nagios does not need a password).

  1. Use “chsh nagios” to set the shell of nagios to “/bin/bash”.
  2. Use “su – nagios” to switch to the home directory of nagios.
  3. Create a key pair if necessary e.g. using
    ssh-keygen -t rsa -b 4096 -C your@email.com

    in the .ssh directory of the nagios user and change the owner to the nagios user if necessary.

  4. Append the contents of the file .ssh/id_rsa.pub to the file .ssh/authorized_keys on the server.
  5. Use “chsh nagios” to set the shell of nagios back to “/bin/false”.

Nagios configuration

Finally you have to configure Nagios. Add this to the commands.cfg file:

define command{
	command_name	check_my_free_space
	command_line	/usr/lib/nagios/plugins/check_by_ssh -H ssh.yourserver.com -l yourusername -C "bash checkfreespace.sh" -t 30
	}

Afterward you have to use this new command in your services file, e.g. add this:

define service {
        service_description             Free webspace;
        check_command                   check_my_free_space
        use                             generic-service;
        host_name  android.calengoo.com;
        notification_interval           0 ; set > 0 if you want to be renotified
        normal_check_interval           60
}

Google’s new OAuth2 mechanism for native apps

So far the OAuth2 authentication could be performed with embedded browsers in iOS and Android. However Google has decided that they want apps to use the default browser instead now. One reason is that the default browser might already know the user due to cookies and can shorten the authentication process. The other reason might … Continue reading “Google’s new OAuth2 mechanism for native apps”

So far the OAuth2 authentication could be performed with embedded browsers in iOS and Android. However Google has decided that they want apps to use the default browser instead now. One reason is that the default browser might already know the user due to cookies and can shorten the authentication process. The other reason might be that apps could have theoretically captured the entered password when using an embedded browser. Anyway to switch an iOS app from the old authentication method using GTMOAuth2ViewControllerTouch to the new using OIDAuthorizationRequest requires only a few steps due to the useful library GTMAppAuth:

  1. Create a file called “Podfile” and use it to download and install GTMAppAuth:
    target 'YourProjectName' do
     platform :ios, '7.0'
     pod 'GTMAppAuth'
    end
  2. Install Cocoapods if necessary and then run
    pod install
  3. Open the created Xcode workspace file. Delete the following old files if you have them:
    GTMGatherInputStream.h
    GTMGatherInputStream.m
    GTMMIMEDocument.h
    GTMMIMEDocument.m
    GTMReadMonitorInputStream.h
    GTMReadMonitorInputStream.m
  4. Add a property to your AppDelegate.h file:
    @property(nonatomic, retain) id currentAuthorizationFlow;
  5. Add a handler to your AppDelegate.m file in the application:openURL:sourceApplication:annotation: method. Put this right at the start of the method:
    if ([_currentAuthorizationFlow resumeAuthorizationFlowWithURL:url]) {
     _currentAuthorizationFlow = nil;
     return YES;
    }
  6. Now you are ready to replace your old authentication code which used GTMOAuth2ViewControllerTouch with the new code (taken partially from the GTMAppAuth website):
    OIDServiceConfiguration *configuration = [GTMAppAuthFetcherAuthorization configurationForGoogle];
    
    OIDAuthorizationRequest *request = [[[OIDAuthorizationRequest alloc] initWithConfiguration:configuration
     clientId:GOOGLE_CLIENT_ID
     clientSecret:GOOGLE_CLIENT_SECRET
     scopes:@[OIDScopeEmail, @"https://www.googleapis.com/auth/calendar"]
     redirectURL:[NSURL URLWithString:@"com.example.yourapp:/oauthredirect"]
     responseType:OIDResponseTypeCode
     additionalParameters:nil] autorelease];
    
     // performs authentication request
     AppDelegate *appDelegate = [[UIApplication sharedApplication] delegate];
     appDelegate.currentAuthorizationFlow =
     [OIDAuthState authStateByPresentingAuthorizationRequest:request
     presentingViewController:self
     callback:^(OIDAuthState *_Nullable authState,
     NSError *_Nullable error) {
     if (authState) {
     // Creates the GTMAppAuthFetcherAuthorization from the OIDAuthState.
     GTMAppAuthFetcherAuthorization *authorization = [[[GTMAppAuthFetcherAuthorization alloc] initWithAuthState:authState] autorelease];
  7. Using the authorization result you can now call your existing handler:
    [self viewControllerFinishedWithAuth:authorization error:error accessToken:authState.lastTokenResponse.accessToken];
     NSLog(@"Got authorization tokens. Access token: %@",
     authState.lastTokenResponse.accessToken);
     } else {
     NSLog(@"Authorization error: %@", [error localizedDescription]);
     }
     }];
    
    }
    
    - (void)viewControllerFinishedWithAuth:(id )auth error:(NSError *)error accessToken:(NSString *)accessToken {
     if (error == nil) {
     // OAuth2 Login was successful
     // Serialize to Keychain
     [GTMOAuth2KeychainCompatibility saveAuthToKeychainForName:keychainNameOAuth2
     authentication:auth];

    Now you can use “accessToken” to access Google services. And due to GTMOAuth2KeychainCompatibility you can keep your existing functions that load the authentication data from the keychain.

  8. In the Info.plist file of your app you have to configure the “com.example.yourapp” URL mentioned above so that the browser can open your app and send you the result:
    CFBundleURLTypes
    
        
            CFBundleTypeRole
            Editor
            CFBundleURLSchemes
            
                com.example.yourapp
            
        
    
    

So that’s all you have to change for an existing iOS app. You can find further information in Google’s blogpost.

Search apps for old iOS versions in the Apple AppStore

Looking for a way to find iOS apps for an older iOS device that does not support the latest iOS version? Here you can search the AppStore and view the release date and the current minimum OS version. Using the release date you can determine if there might be an older version of the app … Continue reading “Search apps for old iOS versions in the Apple AppStore”

Looking for a way to find iOS apps for an older iOS device that does not support the latest iOS version? Here you can search the AppStore and view the release date and the current minimum OS version. Using the release date you can determine if there might be an older version of the app that supports an older iOS version than the current minimum OS version. When installing the app on and old device the AppStore will usually ask you if you would like to install an old version that is compatible to your device.




Version Rating Release date Price Title
{{row.minimumOsVersion}} {{row.averageUserRating}} {{row.releaseDate}} {{row.formattedPrice}} {{row.trackName}}

Using a free SSL certificate from Let’s encrypt with Jetty

Let’s encrypt offers free SSL certificates that can also be used with Jetty. The following steps show how to get an install such a certificate: Installation To install the letsencrypt client on a Linux system, just enter these commands: $ git clone https://github.com/letsencrypt/letsencrypt $ cd letsencrypt $ ./letsencrypt-auto –help Request the certificate To request the … Continue reading “Using a free SSL certificate from Let’s encrypt with Jetty”

Let’s encrypt offers free SSL certificates that can also be used with Jetty. The following steps show how to get an install such a certificate:

  1. Installation
    To install the letsencrypt client on a Linux system, just enter these commands:

    $ git clone https://github.com/letsencrypt/letsencrypt
    $ cd letsencrypt
    $ ./letsencrypt-auto --help
  2. Request the certificate
    To request the certificate run this command:

    ./letsencrypt-auto certonly -d yourdomain.com --webroot -w /var/www/

    To verify the address of your server letsencrypt needs to know the directory of your http server. In this example an Apache is used with the directory /var/www/ (and Jetty runs only on the https port).

  3. Convert the certificate into a format Jetty can use
    openssl pkcs12 -export -in /etc/letsencrypt/live/yourdomain.com/fullchain.pem -inkey /etc/letsencrypt/live/yourdomain.com/privkey.pem -out fullchain_and_key.p12 -name jetty
    keytool -importkeystore -destkeystore keystore_le -srckeystore fullchain_and_key.p12 -alias jetty
    keytool -import -destkeystore keystore_le -file /etc/letsencrypt/live/yourdomain.com/chain.pem -alias root
  4. Install the certificate in Jetty
    Just reference the keystore_le file that you have created in the etc/jetty-ssl.xml file:

    /
    /
    
  5. Stop and restart Jetty

URLEncode Online

Here is a simple form that can URLEncode a given text. This can be useful when programming or writing HTML pages. Text: Convert UrlEncode:

Here is a simple form that can URLEncode a given text. This can be useful when programming or writing HTML pages.

Text:



UrlEncode: