<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Cauldron Lake]]></title><description><![CDATA[Snips, gists and tricks, in no particular order]]></description><link>https://carlomorelli.github.io</link><generator>RSS for Node</generator><lastBuildDate>Mon, 02 Apr 2018 11:47:40 GMT</lastBuildDate><atom:link href="https://carlomorelli.github.io/rss/" rel="self" type="application/rss+xml"/><ttl>60</ttl><item><title><![CDATA[Audit licenses in your Java dependencies]]></title><description><![CDATA[<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>Package management on the JDK since the introduction of Maven and Gradle is a goodness that many other dev platforms still can&#8217;t grasp. I&#8217;m thinking of you, pip+virtualenv hacky weirdness :)</p>
</div>
<div class="paragraph">
<p>Recently as work I was tasked with the generic task of auditing the project code for possible Open Source license violations. At first this raised some eyebrows among us, as one thing that I rarely look at though is what license the library is released under. Also, nowadays we use these very rapid and easy to use frameworks like Spring Boot, where more and more dependencies are pulled without us end-users really controlling them.</p>
</div>
<div class="paragraph">
<p>Unfortunately, for a library user, a license is VERY important. It is also important for the code that you write, but the license of the dependency could even affect the licensing scheme of your own code. I&#8217;ll explain better in a moment.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_the_license_spectrum_is_wide">The license spectrum is wide</h2>
<div class="sectionbody">
<div class="paragraph">
<p>I don&#8217;t want to make a blog post about licenses, I am not an expert. Keeping it simple: beside commercial "Closed Source" code (e.g. the Microsoft Windows code) the Open Source code can have a varying degree of copyleft. There are <em>permissive licenses</em>, like <strong>MIT</strong>, <strong>Apache 2.0</strong>, and <strong>BSD</strong>, which allow any fair use of their code (linked or embedded). There are <em>less permissive licenses</em> like the <strong>LGPL</strong> and the <strong>GPL with classpath exception</strong> that allow the code to be linked by any other code (so in Java terms, you can have dependencies with those licenses), but not embedded (or, more likely copy-pasted).
Finally there are licenses like the <strong>GPL</strong> and the <strong>AGPL</strong> which are considered <em>strong copyleft</em> and viral: if your code links a library with GPL code, then your code too should be licensed as GPL. This of course is acceptable if you are developing a open-source project, but it may be not if you are developing a commercial application.</p>
</div>
<div class="paragraph">
<p>If we are talking about commercial applications, it is not even black and white, it really depends also on many things like: is your application a service, or a product? Is your application residing on the customer premises? is your application charged on usage or as flat? and so on and on.</p>
</div>
<div class="paragraph">
<p>As you can understand the terrain here is very flaky, and <em>many commercial companies try to stay away from GPL and similar strong copyleft code</em>. Moreso, the commercial code embedding or linking open source code is required to attach the Licenses of the dependencies with the delivery software as well. So, you may end up being requested to audit code like it happened to me.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_maven_license_plugin">Maven license plugin</h2>
<div class="sectionbody">
<div class="paragraph">
<p>You would think that managing and auditing licenses in Java would be easy. The natural solution would be to use Maven during the CI builds. And this means, we are looking for a Maven plugin just like Jacoco, CheckStyle or similar.</p>
</div>
<div class="paragraph">
<p>I found out, however, that it is not that easy :). Yes, you can produce a <code>mvn site</code> and retrieve all dependencies and transitive dependencies on your project, including the licenses, and audit that potentially huge list. You can actually easily use the official CodeHaus <strong>maven-license-plugin</strong> for this (pretty fast and complete). This is after all the solution that we adopted at my workplace.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_introducing_licensescan_maven_plugin">Introducing licensescan-maven-plugin</h2>
<div class="sectionbody">
<div class="paragraph">
<p>However, it would be really more useful to have an automated way to alert me if my project was using dependency artifacts with permissive or non permissive licenses and potentially fail the build. Surprisingly, the web was not very generous and I only found out about sparse experiments like <a href="https://github.com/mrice/license-check" class="bare">https://github.com/mrice/license-check</a> and <a href="https://www.ayoy.se/en/blog/2017/10-27-maven-license-plugin/" class="bare">https://www.ayoy.se/en/blog/2017/10-27-maven-license-plugin/</a>. I could not make the first work, as the code seems unmaintained.</p>
</div>
<div class="paragraph">
<p>So i said, let&#8217;s take two pidgeons with one stone: I will produce the plugin that I want and in the meantime I will also learn how to produce a Maven plugin myself.</p>
</div>
<div class="paragraph">
<p>Again, turns out that the Maven documentation to write a plugin is not that easy, and I bounced a lot between the official <code>maven.org</code> documentation and StackOverflow.</p>
</div>
<div class="paragraph">
<p>In the end, I was able to hack something in the last two weeks on my own, that I think does the work. It is very basic, but has parametric configuration and I believe it is easy to use. You just need to follow the directions on the GitHub <a href="https://github.com/carlomorelli/licensescan-maven-plugin" class="bare">https://github.com/carlomorelli/licensescan-maven-plugin</a> Readme file. Here is a summary:</p>
</div>
<div class="paragraph">
<p>1) Put the plugin configuration in your <code>pom.xml</code>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>&lt;plugin&gt;
    &lt;groupId&gt;com.github.carlomorelli&lt;/groupId&gt;
    &lt;artifactId&gt;licensescan-maven-plugin&lt;/artifactId&gt;
    &lt;version&gt;1.1&lt;/version&gt;
    &lt;configuration&gt;
      &lt;printLicenses&gt;true&lt;/printLicenses&gt;
      &lt;blacklistedLicenses&gt;
        &lt;license&gt;GNU Lesser General Public License&lt;/license&gt;
        &lt;license&gt;GNU General Public License, v2.0&lt;/license&gt;
      &lt;/blacklistedLicenses&gt;
      &lt;failBuildOnBlacklisted&gt;true&lt;/failBuildOnBlacklisted&gt;
    &lt;/configuration&gt;
  &lt;/plugin&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>2) Put the JitPack repository in your pluginRepository management block&#8201;&#8212;&#8201;this is required as the plugin is not published on the official Maven Central yet:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>  &lt;pluginRepositories&gt;
    &lt;pluginRepository&gt;
      &lt;id&gt;jitpack.io&lt;/id&gt;
      &lt;url&gt;https://jitpack.io&lt;/url&gt;
    &lt;/pluginRepository&gt;
  &lt;/pluginRepositories&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>Here is the outcome of <code>mvn clean compile</code> for a simple Spring Boot project I was using to test the plugin:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>[warn] BLACKLIST
[warn] -----------------------
[warn] Found 5 violations for license 'GNU Lesser General Public License':
[warn]  - ch.qos.logback:logback-classic:1.1.11:compile
[warn]  - ch.qos.logback:logback-core:1.1.11:compile
[warn]  - org.hibernate:hibernate-core:5.0.12.Final:compile
[warn]  - org.hibernate.common:hibernate-commons-annotations:5.0.1.Final:compile
[warn]  - org.hibernate:hibernate-entitymanager:5.0.12.Final:compile
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.753 s
[INFO] Finished at: 2018-04-01T17:02:01+02:00
[INFO] Final Memory: 37M/430M
[INFO] ------------------------------------------------------------------------</code></pre>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_few_warnings">Few warnings</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Hacking Maven plugins is a strange thing. It feels OK, there is an automatic IoC managed by the Maven toolkit so passing parameters is not very difficult. But the documentation is a little bit lacky, so most of the times you are just guessing. Again, god bless StackOverflow.</p>
</div>
<div class="paragraph">
<p>Also <em>I coded the plugin on Java 1.5 standard</em>. I am not sure if forcing 1.8 would have made things not working on older projects. What I am sure is that once you learn Java 8 streams, <strong>it really sucks</strong> to program with the old for loop style. If you ever take a look at the code, please be gentle with the Mojo code :)</p>
</div>
<div class="paragraph">
<p>Tests? Not yet unfortunately. Maybe in future step releases as I find a good way to test the code. Right now I am experimenting with the official <strong>maven-plugin-test-harness</strong>, which is something very ancient (I mean, it is based on JUnit 3.x &#8230;&#8203; ), but allows to hijack the dependency injection for testing.</p>
</div>
<div class="paragraph">
<p>And by all means, contributions are welcome.</p>
</div>
</div>
</div>]]></description><link>https://carlomorelli.github.io/2018/04/01/Audit-licenses-in-your-Java-dependencies.html</link><guid isPermaLink="true">https://carlomorelli.github.io/2018/04/01/Audit-licenses-in-your-Java-dependencies.html</guid><category><![CDATA[Java]]></category><category><![CDATA[ Maven]]></category><category><![CDATA[ Mojo]]></category><category><![CDATA[ Spring-boot]]></category><dc:creator><![CDATA[Carlo Morelli]]></dc:creator><pubDate>Sun, 01 Apr 2018 00:00:00 GMT</pubDate></item><item><title><![CDATA[Dependency Injection demystified]]></title><description><![CDATA[<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>I&#8217;m going to write a quick walthrough post on what is DI, what problems it is trying to solve, and how to manage it concisely in your project.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_what_does_di_mean">What does DI mean</h2>
<div class="sectionbody">
<div class="paragraph">
<p>In a <a href="https://carlomorelli.github.io/2017/04/23/Easily-designing-the-DB-connection-from-Java.html">previous blog post</a> I discussed how I was moving the database backend from a H2 instance to a fully fledged Pgsql with connection pooling. However I was going to keep the H2 conf for testing, so I end up with a static Factory class that can provide the desired DataSource implementation: <code>Factory::getPostgresDataSource</code> and <code>Factory::getH2DataSource</code>.</p>
</div>
<div class="paragraph">
<p>This DataSource impl is used in the app code. Let&#8217;s assume for a moment that my app runs with a normal p.s.v. <code>main()</code> funcion in the <code>App</code> class. That is, we have something like this:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>public class App {
    public static void main(String... args) {
        DataSource ds = Factory.getH2DataSource();
        // comment previous and uncomment next to switch to Postgres
        //DataSource ds = Factory.getPostgresDataSource();
        ...etc...
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>In this case switching from a DataSource implementation to the other would not be such a pain after all.</p>
</div>
<div class="paragraph">
<p>However, things get complicated when it is not the frontend <code>App</code> class that uses the DataSource object, but some internal class. For example, a common pattern in enterprise Java is to have a Repository class that abstracts data manipulation for the frontend <code>App</code> class&#8201;&#8212;&#8201;there are several reasons why this makes sense, like for example the mantainability according to SOLID design.
So let&#8217;s assume instead that <code>App</code> has a reference to <code>Repository</code> and that it is <code>Repository</code> that needs a <code>DataSource</code> instance. Something like:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>public class App {
    public static void main(String... args) {
        Repository repo = new Repository();
        ...etc...
    }
}

public class Repository {
    private DataSource ds;
	public Repository() {
        ds = Factory.getH2DataSource();
        ...constructor...
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>What happened now, is that the <code>DataSource</code> is not manageable anymore from the frontend. Granted, if we want to change the <code>DataSource</code> impl to be the <code>Factory::getPostgresDataSource</code> instead of the original, we can go to the <code>Repository</code> class and tweak the <code>ds = &#8230;&#8203;</code> line. The problem is that if there are more other "middlemen" in the app structure other than <code>Repository</code>, that need to use a <code>DataSource</code> instance, things will go awry pretty soon: if by mistake we are using, say,  <code>DataSourceImpl1</code> in one part of the code and <code>DataSourceImpl2</code> in another part of the code, what is going to happen?</p>
</div>
<div class="paragraph">
<p>So the only solution is to centralize somewhere the "configuration" of what <code>DataSource</code> implementation we want to use, say, in in the <code>App</code> class itself like we were doing originally, and to <strong>inject</strong> the chosen <code>DataSource</code> implementation from there into <code>Repository</code>. In other words: we have to change the <code>Repository</code> code so that the <code>DataSource</code> <strong>dependency</strong> is <strong>injected</strong> in the constructor by the caller, instead of being created over there:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>public class App {
    public static void main(String... args) {
        DataSource ds = Factory.getH2DataSource();
        Repository repo = new Repository(ds);
        ...etc...
    }
}

public class Repository {
    private DataSource ds;
	public Repository(DataSource ds) {
		this.ds = ds;
        .. constructor...
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>This is basically what <strong>Dependency Injection</strong> is all about: <em>to make code more manageable, we centralize all chosen configuration of interfaces and inject it in cascade through the object constructors.</em></p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_di_and_unit_testing">DI and Unit Testing</h2>
<div class="sectionbody">
<div class="paragraph">
<p>When working in structured manner (the aforementioned SOLID method), DI is very useful for unit testing. We can test separately pieces of code and mock the needed resources. In my case, I want that <code>Repository</code> uses the Postgres d.s. in the real world scenario, but the H2 d.s. in unittests. Unit testing then is relatively easy:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>public class RepositoryTest {
    private DataSource ds;
    private Repository repo;

    @Before
    public void prepare() {
        ds = Factory.getH2DataSource();
    }

    @Test
    public void test1() {
        repo = new Repository(ds);
        ...test content...
    }

    ...other tests...
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Of course it could be the case to totally create a database mock (e.g. using Mockito) instead of relying on H2 for testing. It does not matter though, because in the <code>@Before</code> function we would just write <code>ds = mock(DataSource.class)</code> instead of taking an implementation from <code>Factory</code>, but the injection of <code>ds</code> into <code>Repository</code> would still be done in the same way (the tests would not change&#8230;&#8203; nice!).</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_google_guice">Google Guice</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Dependency Injection as a concept is not difficult to grasp. However, complex projects need to inject multitudes of configurations in multiple different places, and so DI easily becomes a burden.</p>
</div>
<div class="paragraph">
<p>Google Guice to the rescue! Guice is a DI framework oriented to simplicity and in-code configuration <em>through the power of decorators</em>. First, we need to add the dependency to the project <code>pom.xml</code>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>    &lt;dependency&gt;
        &lt;groupId&gt;com.google.inject&lt;/groupId&gt;
        &lt;artifactId&gt;guice&lt;/artifactId&gt;
        &lt;version&gt;4.1.0&lt;/version&gt;
    &lt;/dependency&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>Secondly, with Guice the DI configuration must be done within a class extending the abstract class <code>AbstractModule</code> provided by the framework itself. Let&#8217;s call this class <code>AppConfig</code>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>public class AppConfig extends AbstractModule {
    @Override
    protected void configure() {
        bind(DataSource.class).toInstance(Factory.getH2DataSource());
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>The <code>bind(this).to(that)</code> DSL is very readable and intuitive: the <code>this</code> must be an interface, the <code>that</code> must be any implementation of that interface, that&#8217;s it!</p>
</div>
<div class="paragraph">
<p>More precisely, you have to master few different variants: if you get the implementation from a factory (like in our example), you have to use the <code>.toInstance()</code> DSL, but if you get the implementation from a written class, the correct syntax is <code>.to()</code>.</p>
</div>
<div class="paragraph">
<p>There is a little other step to do in the actual <code>App</code> class code, and is to retrieve the configured implementation in <code>AppConfig</code> within the <code>App</code> class itself, using the <code>Guice::createInjector</code> function:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>public class App {
    public static void main(String... args) {
        DataSource ds = Guice.createInjector(new AppConfig()).getInstance(DataSource.class);
        Repository repo = new Repository(ds);
        ...etc...
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>So the net number of lines has slightly increased, but the DI configuration is now isolated in its own class. If we want to switch from H2 to Postgres, we just go to <code>AppConfig</code> and modify the line in <code>configure()</code> as <code>bind(DataSource.class).toInstance(Factory.getPostgresDataSource());</code> without even touching the Application itself or its dependencies.</p>
</div>
<div class="paragraph">
<p>Needless to say, the very same approach can be done in unit testing: we can create an independent <code>TestConfig</code> class extending <code>AbstractModule</code>, and in the <code>RepositoryTest</code> discussed in the last section we would simply change the line in the <code>@Before</code> function like this:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>    @Before
    public void prepare() {
        ds = Guice.createInjector(new TestConfig()).getInstance(DataSource.class);
    }</code></pre>
</div>
</div>
<div class="paragraph">
<p>and the tests would still work as they did previously, no line of code should be touched other than this.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_foreword">Foreword</h2>
<div class="sectionbody">
<div class="paragraph">
<p>You can see that DI is a very important piece of the puzzle in writing enterprise-quality code. In must be said though that it is a pattern coming from how Java (and C#) are structured and have traditionally been programmed&#8201;&#8212;&#8201;especially considering the evolution of those languagues and what became a pattern (DI) and what became an anti-pattern (not using DI when having multiple implementations of interfaces).</p>
</div>
<div class="paragraph">
<p>Other languages don&#8217;t have interfaces and in those kind of languages DI management is simply impossible (as also mocking is impossible). Some of these, like Python, have evolved in time and even if they don&#8217;t provide interfaces, they provide Abstract Base Classes (ABCs) as injecting point, so DI and mocking <a href="https://www.youtube.com/watch?v=zqRd941NXlI">are possible</a>.</p>
</div>
<div class="paragraph">
<p>Hope you liked my tutorial!</p>
</div>
</div>
</div>]]></description><link>https://carlomorelli.github.io/2017/05/06/Dependency-Injection-demystified.html</link><guid isPermaLink="true">https://carlomorelli.github.io/2017/05/06/Dependency-Injection-demystified.html</guid><category><![CDATA[Java]]></category><category><![CDATA[ Guice]]></category><category><![CDATA[ Maven]]></category><dc:creator><![CDATA[Carlo Morelli]]></dc:creator><pubDate>Sat, 06 May 2017 00:00:00 GMT</pubDate></item><item><title><![CDATA[Portable PostgreSQL distribution for Windows]]></title><description><![CDATA[<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>For a personal project in Java I was using the well-known <a href="http://www.h2database.com/html/main.html">H2 database</a> (which is implemented in Java itself).</p>
</div>
<div class="paragraph">
<p>H2 is a great, easy-to-use db and personally I always like to keep things simple. However, if you are targeting the Cloud, sooner or later you will need advanced features like  networked DB server, user management and  connection pooling, so finally I decided to get a grip and move from H2 to something more Enterprise-level.</p>
</div>
<div class="paragraph">
<p>After some research about the DB platform to adopt, I opted for Postgres. In this post I will show an easy way to deploy an installer-less Postgres on Windows machines&#8201;&#8212;&#8201;something many people on the Internet call 'portable version'.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_install_and_configure_portable_postgres">Install and configure 'portable' Postgres</h2>
<div class="sectionbody">
<div class="paragraph">
<p>On my Windows pc I like to micromanage the system and keep things tidy. If a software I need offers the 'zip' binary archive to download instead of any MSI/EXE installer, I usually go that way.</p>
</div>
<div class="paragraph">
<p>I put normal software blobs in a <code>C:\opt</code> folder, and if it is developement-related software, I use the  <code>C:\workspace\opt</code> folder (as <code>C:\workspace</code> is the root of all my code). Yes yes I know, bad habits from Linux&#8230;&#8203; moving on :P</p>
</div>
<div class="paragraph">
<p>It happens that Postgres offers precisely this: <a href="https://www.enterprisedb.com/download-postgresql-binaries" class="bare">https://www.enterprisedb.com/download-postgresql-binaries</a>. So time to download the <em>Win-x86-64</em> blob for version 9.5.6 and unpack it.</p>
</div>
<div class="paragraph">
<p>The binary blob provides not only the classic Pgsql commands (initdb, dropdb, psql), but also the handy pgAdminIII frontend. In the root folder of the unpacked software (in my case, <code>C:\workspace\opt\postgres-9.5.6\</code>), then, create the batch file <code>launch_pg.bat</code> with this content:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>REM contents
@echo off
set PATH="%~dp0\bin";%PATH%
set PGDATA=%~dp0\data
set PGDATABASE=postgres
set PGUSER=postgres
set PGPORT=5439
set PGLOCALEDIR=%~dp0\share\locale
"%~dp0\bin\initdb" -U postgres -A trust
"%~dp0\bin\pg_ctl" -D "%~dp0/data" -l logfile start
echo "#### Ready. Press space to shutdown..."
pause &gt; null
"%~dp0\bin\pg_ctl" -D "%~dp0/data" stop</code></pre>
</div>
</div>
<div class="paragraph">
<p>If you are wondering, <code>%~dp0</code> variable return the current directory&#8201;&#8212;&#8201;that is, the directory where the .bat file is being executed.</p>
</div>
<div class="paragraph">
<p><strong>WARNING</strong>: The first run of this script will initialize the database in the <code>%PGDATA%</code> directory with the <code>initdb</code> line. <em>For the following launches, remember to comment that line</em>, otherwise you&#8217;ll see some error message popping up.</p>
</div>
<div class="paragraph">
<p>At this point, I am basically done: I&#8217;ll just create Desktop launchers for both this .bat file and the pgAdmin.exe file that you can find in <code>bin\</code>. As you can understand my need is to launch and terminate the server on-demand.</p>
</div>
<div class="paragraph">
<p>However, other people may want to launch Postgres as system service or scheduled task (at startup). If that&#8217;s the case, I suggest to create a startup task in Windows' Task Scheduler in Control Panel.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_what_db_server_to_use_a_small_comparison">What DB server to use? a small comparison</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Few words on why I chose Postgres? Basically, because it is mantained and popular, it is the one I felt more confortable together with MySQL, with the advantage that installing it on Windows is relatively painless&#8201;&#8212;&#8201;as you can see. Also, apparently it is the most lightweight of all: my home machine is not that great&#8201;&#8212;&#8201;it&#8217;s a desktop-replacement laptop from 2012 with Windows 7, so no heavy loads thank you :-).
In general the choice was limited to the followings:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>MySQL / mariaDB</p>
<div class="ulist">
<ul>
<li>
<p>Pro: fast, stable, open-source</p>
</li>
<li>
<p>Con: large installation, complex on Windows</p>
</li>
</ul>
</div>
</li>
<li>
<p>PostgreSQL (aka Postgres, aka Pg)</p>
<div class="ulist">
<ul>
<li>
<p>Pro: the most deployed, documented, and developed; open-source; the installation package is &lt; 100 Mb for version 9.5</p>
</li>
<li>
<p>Con: Not very fast for big transactions</p>
</li>
</ul>
</div>
</li>
<li>
<p>Microsoft SQLServer 2017 Developer</p>
<div class="ulist">
<ul>
<li>
<p>Pro: free, the most enterprise-grade probably</p>
</li>
<li>
<p>Con: super difficult to configure, non open, and with a laaaaarge installation</p>
</li>
</ul>
</div>
</li>
<li>
<p>A NoSQL database</p>
<div class="ulist">
<ul>
<li>
<p>Pro: really fast developing field in the recent years</p>
</li>
<li>
<p>Con: it is a fragmented field, with many products specialized on different aspects (eg. ElasticSearch on clustered responsiveness, MongoDB on data reliability, ArangoDB on other features); also, generally the noSQL dbs are not lightweight</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="paragraph">
<p>The drawbacks of using a SQL RDBMS instead of a NoSQL db is that <em>it is more complicated</em>. You have not only to manage relational data in SQL transactions (which don&#8217;t always map easily to object-oriented patterns, regardless of what you thing of ORM libraries), but also you have to account for Schema formatting and preparation in your app, database migrations and so on.</p>
</div>
<div class="paragraph">
<p>In a next blog post, I will discuss how I manage Postgres (via a connection-pool) using the canonical Java JDBC way. Stay tuned!</p>
</div>
</div>
</div>]]></description><link>https://carlomorelli.github.io/2017/04/23/Portable-PostgreSQL-distribution-for-Windows.html</link><guid isPermaLink="true">https://carlomorelli.github.io/2017/04/23/Portable-PostgreSQL-distribution-for-Windows.html</guid><category><![CDATA[Postgres]]></category><category><![CDATA[ Windows]]></category><dc:creator><![CDATA[Carlo Morelli]]></dc:creator><pubDate>Sun, 23 Apr 2017 00:00:00 GMT</pubDate></item><item><title><![CDATA[Easily designing the DB connection from Java]]></title><description><![CDATA[<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>This blog post is the second part of a series. In the previous, I showed how to install a portable version of Postgres on Windows and which reason led me to chose Postgres (<a href="https://carlomorelli.github.io/2017/04/23/Portable-PostgreSQL-distribution-for-Windows.html">link</a>).</p>
</div>
<div class="paragraph">
<p>All right, so now it&#8217;s the time to spice up with some code examples.</p>
</div>
<div class="paragraph">
<p>First of all, as mentioned at the beginning, the reason I needed a full-featured Db was to have a networked server, and a connection pool under use. By the way, check this out: <a href="http://stackoverflow.com/questions/4041114/what-is-database-pooling">what is connection pooling and why I need it?</a></p>
</div>
<div class="paragraph">
<p>In Java traditionally the JDBC method of connecting to databases is straghtforward, and adding a connection pool is painless. Most of technologies implement standard interfaces from <code>javax.*</code> after all.</p>
</div>
<div class="paragraph">
<p>There are several CP libs out there (the most used seem to be C3PO and Apache Commons DBCP), but I chose to be adventurous and use <a href="https://github.com/brettwooldridge/HikariCP">HikariCP</a>: after all they claim to be more robust and optimized than C3PO so I have the chance to learn to use this lib.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_adding_the_dependencies">Adding the dependencies</h2>
<div class="sectionbody">
<div class="paragraph">
<p>This should appear in the <code>pom.xml</code>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>&lt;dependencies&gt;
    &lt;dependency&gt;
        &lt;groupId&gt;com.zaxxer&lt;/groupId&gt;
        &lt;artifactId&gt;HikariCP&lt;/artifactId&gt;
        &lt;version&gt;2.6.1&lt;/version&gt;
    &lt;/dependency&gt;
    &lt;dependency&gt;
        &lt;groupId&gt;org.postgresql&lt;/groupId&gt;
        &lt;artifactId&gt;postgresql&lt;/artifactId&gt;
        &lt;version&gt;42.0.0&lt;/version&gt;
    &lt;/dependency&gt;
    &lt;dependency&gt;
        &lt;groupId&gt;com.h2database&lt;/groupId&gt;
        &lt;artifactId&gt;h2&lt;/artifactId&gt;
        &lt;version&gt;1.4.192&lt;/version&gt;
    &lt;/dependency&gt;
    &lt;dependency&gt;
        &lt;groupId&gt;org.sql2o&lt;/groupId&gt;
        &lt;artifactId&gt;sql2o&lt;/artifactId&gt;
        &lt;version&gt;1.5.4&lt;/version&gt;
    &lt;/dependency&gt;
&lt;/dependencies&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>You actually only need the Postgresql (which is actually the Postgres JDBC driver for Java) and HikariCP dependencies. The reason why the other dependencies are shown is that the code that I will show you now is also managing the old H2 db instance I had before this change (which I&#8217;m retaining for testing), and uses the library SQL2o for submitting queries&#8201;&#8212;&#8201;more on this in the examples.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_using_hikari">Using Hikari</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Using Hikari is very straightforward. Instead of instantiating a connector to the Postgres JDBC driver directly, we have to instantiate the <code>javax.sql.DataSource</code> implementation of HikariCP: A configuration file in <code>src/main/resources</code> will worry telling Hikari how to connect to Pg and which Driver class to use:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>System.setProperty("hikaricp.configurationFile", "src/main/resources/conf.properties");
DataSource ds = new HikariDataSource();</code></pre>
</div>
</div>
<div class="paragraph">
<p>The config file content should be something like this:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>dataSourceClassName=org.postgresql.ds.PGSimpleDataSource
dataSource.user=postgres
dataSource.password=postgres
dataSource.databaseName=postgres
dataSource.portNumber=5439
dataSource.serverName=localhost</code></pre>
</div>
</div>
<div class="paragraph">
<p>Because I&#8217;m preserving also H2 in my project for testing, I will create a DataSourceFactory class that can return the DataSource implementation that I want, for example using some <strong>Dependency Injection</strong> lib&#8201;&#8212;&#8201;but this is a topic for another time.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>// imports omitted
public class DataSourceFactory {
    public static DataSource getPostgresHikariCPDataSource() throws SQLException {
        System.setProperty("hikaricp.configurationFile", "src/main/resources/configuration.properties");
        return new HikariDataSource();
    }
    public static DataSource getH2DataSource() {
        JdbcDataSource ds = new JdbcDataSource();
        ds.setURL("jdbc:h2:./test");
        ds.setUser("sa");
        ds.setPassword("sa");
        return ds;
    }
}</code></pre>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_the_data_access_object">The Data access object</h2>
<div class="sectionbody">
<div class="paragraph">
<p>What I usually do at this point, is having a a Dao class that masks, or abstracts, the database interactions.
In the future, I want this object Dao to be <strong>injected</strong>, so it will have a constructor that gets a DataSource implementation, coming from the static Factory that we wrote before.
As mentioned, I&#8217;m using Sql2o library which is a nice wrapper of SQL statements in object oriented form (it is NOT an ORM like Hibernate, it is just a SQL formatter).
So the Sql2o main class accepts DataSource injection instead of direct DB connection so we end up with a Dao class like this:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>public class Dao {

    private Sql2o dao;

    public Dao(DataSource ds) {
        LOG.info("Connecting to database [ds='{}']...", ds);
        dao = new Sql2o(ds);
    }

    public List&lt;Person&gt; fetchAll() {
        String sql = "SELECT * FROM people";
        try(Connection conn = dao.open()) {
            return conn.createQuery(sql).executeAndFetch(Person.class);
        }
    }

    public void insert(Person person) {
        String sql = "INSERT INTO person(index,name,surname,address,age) VALUES (:index,:name,:surname,:address,:age)";
        try(Connection conn = dao.open()) {
            conn.createQuery(sql).bind(person).executeUpdate();
        }
    }

    public void flushTable() {
        String sql = "TRUNCATE TABLE person";
        try(Connection conn = dao.open()) {
            conn.createQuery(sql).executeUpdate();
        }
    }

    public void prepareDb() {
         // submit the schema and create table if not availabe
         ...
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Aaaaand, to wrap this tutorial up, here is how you would use the Dao in the caller code:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code>   ...
   public static void main(String...args) throws SQLException {
        Dao dao = new Dao(DataSourceFactory.getPostgresHikariCPDataSource());
        dao.prepareDb();
        IntStream.range(0, 100).forEach( x -&gt; dao.insertItem(new Item(x+1, "test" +x)));
        List&lt;Item&gt; list = dao.fetchAllItems();
        list.forEach(item -&gt; LOG.info("Found item with index {}", item.getIndex()));
    }
    ...</code></pre>
</div>
</div>
<div class="paragraph">
<p>Clearly it is not worth the pain of managing a DB server if you only manage a single kind of entity (<code>Person.class</code>). The focus of this tutorial is not really the complexity of the data to manage, but how to build the persistence layer of your application is a structured manner and to be ready to switch database under the hood&#8201;&#8212;&#8201;and also some advanced information about connection pooling :) Hope you liked it!</p>
</div>
</div>
</div>]]></description><link>https://carlomorelli.github.io/2017/04/23/Easily-designing-the-DB-connection-from-Java.html</link><guid isPermaLink="true">https://carlomorelli.github.io/2017/04/23/Easily-designing-the-DB-connection-from-Java.html</guid><category><![CDATA[Java]]></category><category><![CDATA[ Maven]]></category><category><![CDATA[ Postgres]]></category><category><![CDATA[ HikariCP]]></category><category><![CDATA[ H2]]></category><category><![CDATA[ Sql2o]]></category><dc:creator><![CDATA[Carlo Morelli]]></dc:creator><pubDate>Sun, 23 Apr 2017 00:00:00 GMT</pubDate></item><item><title><![CDATA[Generating Docker artifacts in your Java project]]></title><description><![CDATA[<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>I have been working for 6 months in a new Java/Maven project in my company. It is quite exciting, we do a lot of cutting-edge technology and of course we use massively Jenkins for continous integration.</p>
</div>
<div class="paragraph">
<p>One of the problems that we started facing at one point is <strong>what</strong> to deliver. We were used to publish our WAR artifact to Artifactory, where it unfortunately layed there gaining only dust. Then our integration-test framework would run immediately after the build (triggered by <code>mvn integration-test</code>), and instead of retrieving the artifact, would use a new build run to generate the WAR in the <code>target/</code> dir and start from there.</p>
</div>
<div class="paragraph">
<p>The test framework targets Amazon Web Services and from the Day One we decided to have everything dockerized. So the deployment script would do the following:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>build the WAR file again</p>
</li>
<li>
<p>stream it to the AWS target environment using the target Docker&#8217;s REST API (ugh, really: we streamed a ByteArray of 16 Mb of data&#8201;&#8212;&#8201;more or less&#8201;&#8212;&#8201;in HTTP rest requests)</p>
</li>
<li>
<p>on AWS, pull the Jetty base image from Docker hub</p>
</li>
<li>
<p>on AWS, package a new image</p>
</li>
<li>
<p>on AWS, spawn the container (together together with other ancillary services)</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>This is clearly not very efficient. We have a lot of tools at our avaiablity (Artifactory in primis) but we are not able to make them work in a meaningful way.</p>
</div>
<div class="paragraph">
<p>So I took myself the task to improve our workflow, thanks also to our DevOps specialist who explained me a lot of nuts&#8217;n&#8217;bolts of Docker.</p>
</div>
<div class="paragraph">
<p>It was clear that the best artifact to work with is not a WAR file, but a Docker image itself. I found out the wonderful <strong>Docker-maven-plugin</strong> by <strong>Spotify</strong> (check it out on <a href="https://github.com/spotify/docker-maven-plugin">their github repo</a>) to really suit my need. With it I can create and publish the Docker image of my app directly on Artifactory with a single Maven goal.</p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="_prerequisites">Prerequisites</h3>
<div class="paragraph">
<p>Your Artifactory should have been enabled to run as a "Docker Registry". This basically means that we are able to push and pull images to and from it.</p>
</div>
<div class="paragraph">
<p>The biggest example of a docker registry is the official <a href="https://hub.docker.com/">Docker Hub</a> where lots of mainstream porjects publish their public images.</p>
</div>
<div class="paragraph">
<p>Talk to your network administrator for all the concerns about it. You can also experiment by creating a Docker registry on your local machine while waiting for the IT guys to answer (it usally takes some time :P ). More informations on the Docker registry can be found in the Docker guides.</p>
</div>
<div class="paragraph">
<p>Second big point is, what kind of disk space on Artifactory do you have and what bandwidth availability. A well built Docker image should not require much more than a WAR/JAR artifact takes in terms of size and upload speed, but some tuning may be needed. My suggestion: try, try and retry. Docker is not easy to understand at the beginning.</p>
</div>
</div>
<div class="sect2">
<h3 id="_efficiency_in_docker_images_creation">Efficiency in Docker images creation</h3>
<div class="paragraph">
<p>Long story short: the instruction order in your Dockerfile matters! Every line is a "layer" that Docker creates as basis for the next line, until you reach the end of the Dockerfile and the final image is done.
The Docker Registry you use is a smart service, and if it founds out that some layers that the Docker image you are pushing are already pre-existing in its own belly, it will move on to the next layer (a.k.a. the next line of the Dockerfile) directly. This means that you want to add the WAR/JAR object in the image creation as later as possible, so that a large portion of the final image is in common with other builds of your project.</p>
</div>
<div class="paragraph">
<p>For my project, I use the official Jetty image from the Docker Hub onto which I then put my WAR, and produce a new image that I then upload to our private Artifactory registry.
We achieved a delta factor between a new build and the previous build of about 31 Mbytes, whereas our WAR file weights about 26 Mbytes. Pretty neat!</p>
</div>
<div class="paragraph">
<p>This is what my Dockerfile <strong>would</strong> be:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>FROM jetty:9.3.12-jre8
COPY mywarfile-build12345.war /opt/jetty/webapps/   # this line differentiates the images one another
ENTRYPOINT ["java", "-jar", "/opt/jetty/start.jar"]</pre>
</div>
</div>
<div class="paragraph">
<p>You will see now how to produce the Docker image without needing any Dockerfile at all using Spotify&#8217;s Docker maven plugin.</p>
</div>
</div>
<div class="sect2">
<h3 id="_configuration">Configuration</h3>
<div class="paragraph">
<p>Let&#8217;s see how to configure the plugin in the project&#8217;s <code>pom.xml</code>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-xml" data-lang="xml">&lt;plugin&gt;
  &lt;groupId&gt;com.spotify&lt;/groupId&gt;
  &lt;artifactId&gt;docker-maven-plugin&lt;/artifactId&gt;
  &lt;version&gt;0.4.13&lt;/version&gt;
  &lt;configuration&gt;
    &lt;imageTags&gt;
      &lt;imageTag&gt;${project.version}-${myimage.build}&lt;/imageTag&gt;
    &lt;/imageTags&gt;
    &lt;imageName&gt;artifactory.mycompany.net/applications/myapp&lt;/imageName&gt;
    &lt;baseImage&gt;jetty:9.3.12-jre8&lt;/baseImage&gt;
    &lt;entryPoint&gt;["java", "-jar", "/opt/jetty/start.jar"]&lt;/entryPoint&gt;
    &lt;workdir&gt;/opt/jetty&lt;/workdir&gt;
    &lt;exposes&gt;8080&lt;/exposes&gt;
    &lt;resources&gt;
      &lt;resource&gt;
        &lt;targetPath&gt;/opt/jetty/webapps/&lt;/targetPath&gt;
        &lt;directory&gt;${project.build.directory}&lt;/directory&gt;
        &lt;include&gt;${project.build.finalName}.war&lt;/include&gt;
      &lt;/resource&gt;
    &lt;/resources&gt;
    &lt;serverId&gt;server_entry&lt;/serverId&gt;
    &lt;registryUrl&gt;https://artifactory.mycompany.net&lt;/registryUrl&gt;
  &lt;/configuration&gt;
&lt;/plugin&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>As you can see the parameters resemble pretty much the intended Dockerfile.</p>
</div>
<div class="paragraph">
<p>Notice that the <strong>server_entry</strong> is the label of an entry that you have to insert into your local <code>~/.m2/settings.xml</code> as such:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-xml" data-lang="xml">&lt;server&gt;
  &lt;id&gt;server_entry&lt;/id&gt;
  &lt;username&gt;insert_login_name_here&lt;/username&gt;
  &lt;password&gt;insert_password_here&lt;/password&gt;
  &lt;configuration&gt;
    &lt;email&gt;insert_your_email_here&lt;/email&gt;
  &lt;/configuration&gt;
&lt;/server&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>Strangely enough the email field is <strong>mandatory</strong> for the current version of the spotify Docker-maven-plugin to work correctly, so watch out!</p>
</div>
</div>
<div class="sect2">
<h3 id="_usage">Usage</h3>
<div class="paragraph">
<p>You can see that what I put in the pom.xml is parametrized by some automatic Maven variables (they look like <code>${something}</code>).
There is a residual parameter which is <code>${myimage.build}</code> which in my case does not come from Maven internals, but from the command line: you can run the docker image generation like this:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>mvn docker:build -Dmyimage.build=12345</pre>
</div>
</div>
<div class="paragraph">
<p>and this will result in a Docker image created locally with tag</p>
</div>
<div class="listingblock">
<div class="content">
<pre>artifactory.mycompany.net/applications/myapp:0.0.1-SNAPSHOT-12345</pre>
</div>
</div>
<div class="paragraph">
<p>(You can check this out with <code>docker ps -a</code> command). The <code>0.0.1-SNAPSHOT</code> part here simply comes from the version of your project (on top of your <code>pom.xml</code> file), so it can be totally different in your case, of course.</p>
</div>
<div class="paragraph">
<p>To push this image to artifactory, we simply need to add a push command to the command line:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>mvn docker:build -Dmyimage.build=12345 -DpushImage</pre>
</div>
</div>
<div class="paragraph">
<p>That&#8217;s it.</p>
</div>
<div class="paragraph">
<p>If you use Jenkins CI like in my case, it is easy to inject the <code>${myimage.build}</code> value from the Jenkins built-in variable <code>$BUILD_NUMBER</code>, so every successful job run will automatically push a different image version on Artifactory. Or you could just forget about it and remove the versioning entirely, and only use the <code>latest</code> tag (this is automatically managed by the Docker-maven-plugin again). Your choice :)</p>
</div>
<div class="paragraph">
<p>Have fun!</p>
</div>
</div>]]></description><link>https://carlomorelli.github.io/2016/10/19/Generating-Docker-artifacts-in-your-Java-project.html</link><guid isPermaLink="true">https://carlomorelli.github.io/2016/10/19/Generating-Docker-artifacts-in-your-Java-project.html</guid><category><![CDATA[Java]]></category><category><![CDATA[ Docker]]></category><category><![CDATA[ Maven]]></category><category><![CDATA[ Spotify]]></category><dc:creator><![CDATA[Carlo Morelli]]></dc:creator><pubDate>Wed, 19 Oct 2016 00:00:00 GMT</pubDate></item><item><title><![CDATA[Hello]]></title><description><![CDATA[<div class="paragraph">
<p>So finally I get my developer&#8217;s blog.</p>
</div>
<div class="paragraph">
<p>I will be using <a href="https://www.hubpress.io">Hubpress.io</a>. It is a cool concept, let&#8217;s see how it goes.</p>
</div>
<div class="paragraph">
<p>Stay tuned.</p>
</div>]]></description><link>https://carlomorelli.github.io/2016/10/18/Hello.html</link><guid isPermaLink="true">https://carlomorelli.github.io/2016/10/18/Hello.html</guid><category><![CDATA[none]]></category><dc:creator><![CDATA[Carlo Morelli]]></dc:creator><pubDate>Tue, 18 Oct 2016 00:00:00 GMT</pubDate></item></channel></rss>