Kotlin Native serverless tutorial: Fun with Fibonacci

Previously, we covered a Kotlin Serverless Function using Apache OpenWhisk. However, since we are using Kotlin, we can make our example a native application that will not run in the JVM. Like before, we will need OpenWhisk as well as Vagrant and Virtual Box.

The complete example can be found in this repository.

Kotlin native serverless

To setup the environment just:

$ git clone --depth=1 https://github.com/apache/incubator-openwhisk.git openwhisk
$ cd openwhisk/tools/vagrant
$ ./hello 

Now we could ssh to the machine with

$ vagrant ssh 

To test the action directly, just use the Docker container that I’ve upload to docker hub:

$ wsk action create native-fibonacci --docker jamedina/kotlin-native-fibonacci 

To test the new action, just run:

$ wsk action invoke --result native-fibonacci --param numbers 5 

This will output something like:

{
  "result": [
      1,
      1,
      2,
      3,
      5
  ]
} 

OpenWhisk allows us to use a docker container that contains a binary that will be executed when an action is invoked. So, the binary will receive 1 argument with a JSON string and it will need to return a JSON string in a single line.

So, to create our function first, we need to have kotlin-native installed and configured:

$ git clone --depth 1 https://github.com/JetBrains/kotlin-native.git /kotlin-native
$ cd /kotlin-native
$ ./gradlew dependencies:update
$ ./gradlew dist
$ export PATH="/kotlin-native/dist/bin:$PATH"
$ kotlinc 

Take care: this last line will cause an error as we haven’t provided any parameters to the compiler, but we’re just doing this in order to download the compiler dependencies and toolchain.

Now we will create our project, a native use of kotlin with a gradle plugin. The initial set up goes like this:

buildscript {
    repositories {
        mavenCentral()
        maven {
            url "https://dl.bintray.com/jetbrains/kotlin-native-dependencies"
        }
    }

    dependencies {
        classpath "org.jetbrains.kotlin:kotlin-native-gradle-plugin:+"
    }
}

apply plugin: 'konan'

konanArtifacts {
    fibonacci {
    }
} 

Now, we need to specify where the kotlin-native is installed using the gradle.properties:

konan.home=/kotlin-native/dist 

Now, let’s just create one simple main in the folder src/main.kt:

fun fibonacci(numbers: Long): Array<Long> {
  if (numbers == 0L) return arrayOf(0L)
  if (numbers == 1L) return arrayOf(1L)

  var previous = 1L
  var current = 1L
  var temp: Long

  return arrayOf(1L, 1L) + (1..(numbers - 2)).map {
    temp = current + previous
    previous = current
    current = temp
    current
  }.toList().toTypedArray()
}

fun main(args: Array<String>) = fibonacci(5).forEach(::println) 

Now, to compile:

$ gradlew clean build 

We use clean build because sometimes gradle will not compile the files even with the files changed.

Then, we could run it with:

$ ./build/konan/bin/fibonacci.kexe
1
1
2
3
5 

Now, we need to use JSON. Since we aren’t using the JVM in this build or any Java code, we are just going to use a C JSON library. I like parson for this, since I think it suits this task perfectly.

First clone it, or create a submodule in your git:

$ mkdir src/cpp
$ cd src/cpp
$ git clone --depth 1 https://github.com/kgabis/parson.git 

Now, I’ve created a shell script to compile the C code into a library that we could use later:

DEPS=$(dirname `type -p konanc`)/../dependencies

if [ x$TARGET == x ]; then
case "$OSTYPE" in
  darwin*)  TARGET=macbook ;;
  linux*)   TARGET=linux ;;
  *)        echo "unknown: $OSTYPE" && exit 1;;
esac
fi

CLANG_linux=$DEPS/clang-llvm-3.9.0-linux-x86-64/bin/clang++
CLANG_macbook=$DEPS/clang-llvm-3.9.0-darwin-macos/bin/clang++

var=CLANG_${TARGET}
CLANG=${!var}

mkdir -p build/clang/

$CLANG -x c -c src/main/cpp/parson/parson.c -o build/clang/parson.bc -emit-llvm || exit 1 

This script will create a library in LLVM format, which is the architecture used by Kotlin Native.

Now we need to update our gradle script to compile the C code, and to link it to our program.

buildscript {
    repositories {
        mavenCentral()
        maven {
            url "https://dl.bintray.com/jetbrains/kotlin-native-dependencies"
        }
    }

    dependencies {
        classpath "org.jetbrains.kotlin:kotlin-native-gradle-plugin:+"
    }
}

apply plugin: 'konan'

konanInterop {
    Parson {
        includeDirs "${project.projectDir}/src/main/cpp/parson"
    }
}

konanArtifacts {
    fibonacci {
        useInterop "Parson"
        nativeLibrary "${project.buildDir.canonicalPath}/clang/parson.bc"
    }
}

task compileCpp(type: Exec) {
    dependsOn 'genParsonInteropStubs'
    workingDir project.getProjectDir()
    commandLine './buildCpp.sh'
}

compileKonanFibonacci {
    dependsOn 'compileCpp'
}

Finally, we need to add a definition file for the C code so Kotlin could create and stub for it.

The file should be place in src/c_interop/Parson.def:

headers = parson.h
headerFilter = parson.h 

Now, we can just build the same as before to check that everything is ok:

$ gradlew clean build 

This should generate some files including:

• build/clang/parson.bc – This is the linked library.
• build/konan/interopStubs/genParsonInteropStubs/Parson/Parson.kt – This is the Kotin Stub used for calling the library.

Now, we modify our program to use parson:

import kotlinx.cinterop.*
import Parson.*

fun fibonacci(numbers: Long): Array<Long> {
  if (numbers == 0L) return arrayOf(0L)
  if (numbers == 1L) return arrayOf(1L)

  var previous = 1L
  var current = 1L
  var temp: Long

  return arrayOf(1L, 1L) + (1..(numbers - 2)).map {
    temp = current + previous
    previous = current
    current = temp
    current
  }.toList().toTypedArray()
}

fun Array<String>.paramOrElse(name: String, elseValue: Long): Long {
  var result = elseValue
  if (this.size > 0) {
    val json = this[0]
    memScoped {
      val schema = json_parse_string(json)
      if (schema != null) {
        val root = json_object(schema)
        if (root != null) {
          if (json_object_has_value(root, name) == 1) {
            result = json_object_get_number(root, name).toLong()
          }
        }
        json_value_free(schema)
      }
    }
  }
  return result
}

fun Long.throughFunction(operation: (Long) -> Array<Long>): String {
  var result = "{}"
  val elements = operation(this)
  memScoped {
    val schema = json_value_init_object()
    val root = json_value_get_object(schema)

    json_object_set_value(root, "result", json_value_init_array())
    val array = json_object_get_array(root, "result");

    elements.forEach {
      json_array_append_number(array, it.toDouble())
    }

    result = json_serialize_to_string(schema)?.toKString()!!
    json_value_free(schema)
  }
  return result
}

fun main(args: Array<String>) {

  val result = args.paramOrElse("numbers", 10L).throughFunction(::fibonacci)

  println(result)
} 

We have created a couple of functions that uses the Kotlin/Native interoperability so we could invoke Parson.

After building the program again, we can just use it like this:

$ ./build/konan/bin/Fibonacci.kexe "{ \"numbers\" : 5 }"
{"result":[1,1,2,3,5]} 

Now, we need to create a docker for it. OpenWhisk has a docker base image that is based on alpine linux. I’ve not been able to make Kotlin-Native to work on alpine, so I’ve create a base docker image that anyone can use for making Kotlin Native Serverless functions.

The docker file for that image is:

# Dockerfile for runing a openwishk Kotlin native action
FROM openjdk:8-jdk

#install kotlin native
RUN git clone --depth 1 https://github.com/JetBrains/kotlin-native.git
WORKDIR kotlin-native
RUN ./gradlew dependencies:update
RUN ./gradlew dist

ENV PATH /kotlin-native/dist/bin:$PATH

#install python
RUN apt-get update
RUN apt-get install python3-setuptools -y
RUN easy_install3 pip
RUN pip3 install --upgrade pip setuptools six
RUN pip3 install --no-cache-dir gevent==1.2.1 flask==0.12

#install c libraries
RUN apt-get update
RUN apt-get install libc-dev -y

#preparing the action proxy
ADD actionProxy/ /actionProxy
WORKDIR /actionProxy
ENV FLASK_PROXY_PORT 8080

CMD ["/bin/bash"]

I’ve used openjdk 8 since we are going to use gradle in our build. It uses the actionProxy python3 script that is provide by OpenWhisk to create a flask server that will serve our action trough HTTP. Our actions should be placed in the folder /action and be named exec.

So now we could create a docker image for our fibonacci function:

# Dockerfile for runing a Kotlin native fibonacci action
FROM jamedina/openwhisk-kotlin-native

#add basic gradle files
ADD .gradle/ /temp-build/.gradle
ADD gradle/ /temp-build/gradle
ADD build.gradle /temp-build
ADD gradlew /temp-build
ADD gradle.properties /temp-build

#get the compiler
WORKDIR /temp-build
RUN ./gradlew

# add our action code
ADD src/ /temp-build/src
ADD buildCpp.sh /temp-build

#build our action
RUN ./gradlew build

#copy the action as the default OpenWhisk docker actions
RUN mkdir /action
RUN cp /temp-build/build/konan/bin/fibonacci.kexe /action/exec

#clean up
RUN rm -rf /temp-build
RUN rm -rf /kotlin-native

CMD ["/bin/bash", "-c", "cd /actionProxy && python3 -u actionproxy.py"] 

Finally, I create a simple scripts to publish the images in docker hub. Remember to login before!

#!/usr/bin/env bash
echo "creating base image jamedina/openwhisk-kotlin-native"
docker build -t jamedina/openwhisk-kotlin-native docker/

if [ "$?" -eq "0" ]
then
  echo "pushing jamedina/openwhisk-kotlin-native"
  docker push jamedina/openwhisk-kotlin-native
  if [ "$?" -eq "0" ]
  then
    echo "creating image jamedina/kotlin-native-fibonacci"
    docker build -t jamedina/kotlin-native-fibonacci .
    if [ "$?" -eq "0" ]
    then
      echo "pushing"
      docker push jamedina/kotlin-native-fibonacci
      if [ "$?" -eq "0" ]
      then
        echo "done"
      else
        echo "fail to push fibonacci docker"
      fi
    else
      echo "fail to build fibonacci docker"
    fi
  else
    echo "fail to push base docker"
  fi
else
  echo "fail to build base docker"
fi 

In the same way that the files are added to the docker, we can just run that command to get update the image. This will only change some of the parts that needed it.

So, now we could put all things together to do:

$ ./build.sh
$ wsk action create native-fibonacci --docker jamedina/kotlin-native-fibonacci
$ wsk action invoke --result native-fibonacci --param numbers 5
{
  "result": [
      1,
      1,
      2,
      3,
      5
  ]
}

This has been a great example to learn Kotlin Native, so sure I’ll do more things with it in the future!

 

This post was originally published on Juan Medina’s blog.