Glen Mazza's Weblog

Main | Next page »

https://web-gmazza.rhcloud.com/blog/date/20160624 Friday June 24, 2016

TightBlog: June 2016 Update

Continued work on the TightBlog front-end. While TightBlog is not a single-page app, AngularJS is nonetheless playing more and more of a role on the front end for its two-way binding. Backend, I've updated to Java 8 time as well as upgraded to Struts 2.5. About 60% of the UI is now REST-based. I'm expecting "soonish" to make an testing release that I can place my own blog on while continuing to do front-end enhancements. In particular, I'd like to get the comment management screen more streamlined.

Stats: Eight fewer Java source files, moving me down to 206, 58% less than Roller 5.1.2. 52 JSPs vs. 92 in Roller, but 10-20 more JavaScript files on my side. The simplifications altogether have resulted in about 66.8K lines of source code removed. I'm increasingly running into brick walls in trying to simplify the application further, a good sign that my refactoring work is coming to a close. Nine issues open, with 197 closed.

https://web-gmazza.rhcloud.com/blog/date/20160524 Tuesday May 24, 2016

TightBlog: May 2016 Update

Heavily working on the front-end, moving TightBlog from a server-side rendering with Struts2 & JSPs to a REST-ful architecture. The Spring Framework is doing wonders in allowing me to easily switch to REST API calls, code is nicely melting away in the process. I've mostly finished updating the server admin screens using JQuery and JSRender/JSViews. I'm trying to find a good point right now where I can do a release, deploy my own blog on it, and then continue on with enhancements & refactoring.

Stats: Due to more code auditing and analysis, the issues have jumped up to 20 open vs 177 closed, but most do not need to be done for a first release. Seventeen more Java source files have been removed (actually 20 removed and 3 new ones), dropping me to 214 total. GitHub is reporting I've removed more than 62,000 lines from the project; the OpenHub chart, which sizes the project differently, has noted the project has fallen from about 139K lines of code to 87K since I forked Apache Roller last May. As perhaps a sign of TightBlog's increased modernization, OpenHub is noting the code is 18.9% JavaScript compared to 11% for Roller.

https://web-gmazza.rhcloud.com/blog/date/20160423 Saturday April 23, 2016

TightBlog: April 2016 Update

A satisfyingly productive month. In particular, the blog page rendering system as well as the default blog templates supplied with TightBlog were cleaned out and nicely simplified. At this time, the application is more or less usable as-is, however, I'll be rechecking the data model as well as making an additional pass through the source files to see if I've missed anything.

Stats: Sixteen more Java source files removed, dropping TightBlog to 231 total. From my commit totals, over 55,000 lines removed from the project through 248 non-merge commits. Seven issues open and 157 closed.

https://web-gmazza.rhcloud.com/blog/date/20160330 Wednesday March 30, 2016

TightBlog: March 2016 Update

Did not get as much done the past month due to work constraints, main gain was in upgrading from Log4j to Log4j2. Right now I'm revamping the template customization functionality, once done I have a modest amount of other UI improvements to put in for the first release. Twelve issues open, 140 closed, 5 more Java source files gone (247 for TightBlog vs. 493 for Roller), and inching towards 50,000 lines (49.2K) removed.

https://web-gmazza.rhcloud.com/blog/date/20160224 Wednesday February 24, 2016

TightBlog: February 2016 Update

Still actively working on TightBlog, but due to time constraints could not work much on the project the first couple of weeks of this period. Mainly internal refactoring, including work on simplifying and consolidating the Pager classes that are used to scroll through lists of blog entries, users, comments, weblogs, etc. I also simplified the former roller.properties to mostly just the common fields that people normally customize (file is now about two-thirds smaller, compare TightBlog's to Roller's). For the many potentially useful but seldom overridden configuration settings, those can now be modified via a Spring XML configuration file placed in the same folder as the user's tightblog-custom.properties file. For the rarely overridden settings, using Spring configuration over a property file nicely simplifies the internal code while not placing much of an additional burden on the relative few who will need to override those settings.

Metrics: 14 issues still open, 138 closed, 199 non-merge (i.e, "real") commits with roughly 47,000 more lines removed than added (as shown on the GitHub Contributors tab), and eight more Java source files removed, dropping TightBlog to 252 total.

https://web-gmazza.rhcloud.com/blog/date/20160124 Sunday January 24, 2016

TightBlog: January 2016 Update

Still plugging along, with particular gains in the Spring configuration, page caching, and blog model used by the Velocity rendering. One functional improvement was to pull out the Google Analytics tracker code on blog author previews so they won't show up on GA stats pages. The GitHub Issues List notes 16 issues open and 133 closed. Of the Java source files, 16 more have been removed, dropping TightBlog to 260 classes compared to the 493 in Apache Roller I forked it from. The contributors page notes almost 46,000 lines have been removed from the codebase through 188 commits.

https://web-gmazza.rhcloud.com/blog/date/20151224 Thursday December 24, 2015

TightBlog: December 2015 update

Since my update last month, I've been able to pull out an additional 21 Java source files, dropping TightBlog to 276 total compared to Roller's 493, a nice 44% drop. Other files, such as XMLs and JSPs have also been removed if not to the same degree, as many of those simplifications I had already gotten done while on the Roller team. The GitHub Issues List notes 121 issues closed and 18 still open. Mainly still code refactoring (I'm shifting more to optimizing the SQL queries now -- identifying and removing redundant/unnecessary calls, inefficient calls, etc.) Biggest functional improvement has been in field validation and input field whitespace trimming. I also have been following changes to the Apache Roller project, incorporating those I like into Tightblog as well.

https://web-gmazza.rhcloud.com/blog/date/20151213 Sunday December 13, 2015

Working with TightBlog source code

Note: TightBlog is an unfinished fork of Apache Roller without full rebranding having been finished, so there remain several Roller references below.

This entry explains how to set up a development environment and process for working on the Java-based TightBlog blog server. The tools below are the ones I use, some are optional some (such as preferred IDE or database to use) can be modified to your preference.

Tool list for programming TightBlog:

ToolBefore proceeding, make sure you can...
JDK 1.8Activate "java" and "javac" from any folder in a command-prompt window
Intellij IDEA (I use the free community edition)Can activate the application from a command-line.
Apache MavenActivate "mvn" from any folder in a terminal window
Command-line git clientRun "git" from any folder in a terminal window
Standalone Tomcat 8Can start and view Tomcat from http://localhost:8080
Apache Derby databaseCan activate ij (for Derby) from any folder in a command-prompt window. Note: See below instructions for using MySQL or PostgreSQL instead.
Firefox Web BrowserOptional; used for running Selenium tests without needing to activate Tomcat
SQL Query tool such as SquirrelSQLOptional but recommended; used for querying the TightBlog database instance.
  1. Fork TightBlog and add any desired additional themes - Do a git clone git@github.com:gmazza/tightblog.git from the command-line window. You can also make a fork within GitHub to work from your own account. If you have any of custom blog themes to add, place them in the src/main/webapp/themes folder prior to building TightBlog. Make sure TightBlog build (including JUnit tests) runs successfully before proceeding further.

  2. Prepare database and (optional) mail configuration - Follow the short Chapter 5 and Chapter 6, Sections 1-4 of the Roller Install Guide (ODT). Chapter 5 provides the few commands needed to configure an empty MySQL, PostgreSQL, or Derby database and Chapter 6 the configuration of the roller-custom.properties file, optional Mail configuration, and the required Mail and JDBC JARs that will be needed in Tomcat's lib folder. The test roller-config.properties file that I have in my Tomcat lib folder is as follows:

    # Any properties placed below overrides TightBlog default settings defined in
    # the roller.properties file embedded within the TightBlog WAR, and should be 
    # stored in the CATALINA_HOME/lib folder for Tomcat.
    
    installation.type=auto
    mediafiles.storage.dir=/home/gmazza/work/tightblog-dbs/mediafiles
    search.index.dir=/home/gmazza/work/tightblog-dbs/searchindex
    
    # TightBlog file and logging level
    log4j.appender.roller.File=${catalina.base}/logs/roller-tomcat.log
    log4j.logger.org.apache.roller=INFO
    
    #Derby
    database.configurationType=jdbc
    database.jdbc.driverClass=org.apache.derby.jdbc.ClientDriver
    database.jdbc.connectionURL=jdbc:derby://localhost:1527//home/gmazza/work/tightblog-dbs/MYROLLERDB
    database.jdbc.username=app
    database.jdbc.password=app
    
    # EclipseLink debugging (filepath cannot be simplified via ${catalina.base} as above)
    eclipselink.logging.file=/home/gmazza/work/apache-tomcat-8.0.30/logs/eclipselink-tomcat.log
    eclipselink.logging.level=FINER
    
    # Mail config (See Roller Install Guide)
    mail.configurationType=jndi
    mail.jndi.name=mail/Session
    

    Before moving on to the next step, best to deploy the app/target/roller.war to your Tomcat webapps directory, start Tomcat and confirm you can run the application at http://localhost:8080/roller. This is an important one-time check to ensure your database, roller-custom.properties file, Tomcat, and Roller WAR are all properly configured, and once confirmed you should be in good shape for all subsequent coding and debugging. If any deployment problems, make sure you've reviewed Chapters 5 and 6 of the Install Guide.

  3. Create a script to start development with everything needed - For efficient start-up I've created a hacktb.sh script that opens up all needed windows and applications at once instead of having me needing to do so manually each time I start development. A simplified version of my hacktb.sh is as follows and explained below:

    hackroller.sh:

    gnome-terminal --geometry=132x24 \
       --tab-with-profile=HasTitle --title "TB Trunk" --working-directory ~/work/tightblog \
       --tab-with-profile=HasTitle --title "TB Trunk2" --working-directory ~/work/tightblog -e "bash -c \"echo -Dmaven.surefire.debug; exec bash\"" \
       --tab-with-profile=HasTitle --title "Servlet Container" \
       --tab-with-profile=HasTitle --title "IntellijIDEA" -e "bash -c \"sh idea*/bin/idea.sh; exec bash\"" \
       --tab-with-profile=HasTitle --title "Derby Network" --working-directory $DERBY_HOME/bin -e "bash -c \"startNetworkServer; exec bash\"" \
       --tab-with-profile=HasTitle --title "GEdit" -e "gedit worklog.txt hacktb.sh $CATALINA_HOME/lib/roller-custom.properties $CATALINA_HOME/logs/catalina.out $CATALINA_HOME/logs/roller-tomcat.log `find $CATALINA_HOME/logs/localhost*.log` $CATALINA_HOME/logs/eclipselink-tomcat.log tightblog/app/target/roller.log tightblog/app/target/eclipselink.log " \
       --tab -e "dolphin --geometry=600x500+1+1 tightblog-dbs tightblog $CATALINA_HOME" \
       --tab -e "google-chrome https://github.com/gmazza/tightblog/issues http://localhost:8080/roller" \
       --tab-with-profile=HasTitle --title "SquirrelSQL" --working-directory ~/work/squirrel-sql-3.7 -e "bash -c \"sh squirrel-sql.sh; exec bash\""
    

    The above script opens a multitab Console window which in turn opens a few separate application windows. In particular:

    • I open two terminal tabs pointing to the TightBlog code. This allows me to build TightBlog from one terminal while reviewing my changes (using commands git status and git diff, etc.) in the second window. I echo the -Dmaven.surefire.debug in the second tab as a reminder of the Maven string to add during JUnit test debugging (covered below).
    • I have a separate "ServletContainer" tab open to my home folder for manual activation of servlet containers (Tomcat normally) or other applications from the command line. I don't need this as often due to the copy.sh script (discussed below) I now use.
    • I use a tab for starting IntelliJ IDEA.
    • I use Derby's Network mode (startNetworkServer) instead of its Embedded mode so the database can be accessed by multiple JVM's (one by Tomcat and the other by Squirrel SQL).
    • I open the multi-tab GEdit text editor with several files that are handy for me during development:
      • worklog.txt - A scratchpad file of TODO notes, commands, etc. that I keep for personal use between coding sessions.
      • hacktb.sh - The very script above, I keep handy in case I need to make adjustments or additions to it.
      • roller-custom.properties - The configuration file discussed above and used by all TightBlog deployments to my standalone Tomcat. Changes I made here are activated on the next Tomcat restart.
      • catalina.out, roller-tomcat.log, localhost*.log, eclipselink-tomcat.log - These are logging files filled by Tomcat and/or TightBlog during running, usually very important for debugging as error logging usually gets written to one of these files. When you see terse "System" or similar errors reported by TightBlog in the web browser be sure to check these files for the error in detail.
      • roller.log, eclipselink.log - These files are under TightBlog's target directory and are populated during JUnit test running via the Maven test phase, as discussed below.
    • Dolphin - A multitab file browser that I have opened to the TightBlog trunk code, database files, and Tomcat folders for easy access to files.
    • Firefox - Here I pre-open tabs to the TighBlog website and default webpage location for TightBlog once it's deployed to Tomcat.
    • SquirrelSQL - Commented-out by default but used when I wish to query the TightBlog database to see the values that TightBlog is reading/writing. SquirrelSQL will need your database's JDBC JAR added to it as well as have the connection information configured in roller-custom.properties.

    For scripts such as the above make sure there's no whitespace after the ending "\" on each line (error messages will pop up otherwise), and use a leading # (as shown above for SquirrelSQL) for actions you wish to disable by default. The $CATALINA_HOME specified above, as usual, is the base folder for your standalone Tomcat installation. In addition to the above configuration, you may wish to keep handy the Roller trunk documentation, kept in OpenOffice format in the Roller/docs folder or online.

  4. How to build and deploy to local Tomcat - After a successful mvn clean install from the TightBlog trunk folder, I run this copy.sh shell script that I keep in that folder:

    fuser -k 8080/tcp 8009/tcp 8005/tcp
    rm -r $CATALINA_HOME/webapps/roller
    cp ./app/target/roller.war $CATALINA_HOME/webapps
    rm $CATALINA_HOME/logs/*.log
    rm $CATALINA_HOME/logs/catalina.out
    $CATALINA_HOME/bin/startup.sh
    

    For rapid iteration when I don't need to run the tests each time, I simplify my build and deploy process to a single line: mvn clean install -Dmaven.test.skip ; sh copy.sh.

    The above file first kills the Tomcat instance (via fuser -k), deletes the previous expanded TightBlog webapp directory on the Tomcat instance and copies the latest created Roller WAR over, and clears all the logs before finally restarting Tomcat with the new WAR. The new TightBlog will be accessible at http://localhost:8080/roller again using the same $CATALINA_HOME/lib/roller-config.properties and database (i.e., all database-stored information including test blog data created will be immediately picked up by the new Roller WAR.)

  5. How to debug on local Tomcat - Learning to debug from your IDE any webapp running locally on standalone Tomcat is frequently vital when troubleshooting and thankfully simple to do. First, add to your Tomcat CATALINA_OPTS environment variable:

    export CATALINA_OPTS=$CATALINA_OPTS" -Xdebug -Xrunjdwp:transport=dt_socket,address=5005,server=y,suspend=n"
    

    Then from IntelliJ IDEA (or Eclipse), go to menu item Run -> Edit Configurations, add a Remote Configuration with a name of your choice and accepting the given defaults, and select OK. If TightBlog is already running on Tomcat, debugging can be activated at any time by setting breakpoints in the code that the TightBlog app would be activating, selecting menu item Run --> Debug {debug config name}, and then proceed with any needed code tracing.

  6. How to debug unit tests - To activate a specific TightBlog JUnit test, navigate to the tightblog/app folder and run mvn test -Dtest=TestClassName#OptionalTestMethodName, as explained in the Maven Surefire Plugin documentation. To debug (code trace) the JUnit test within your IDE, add the -Dmaven.surefire.debug setting, set breakpoints in your IDE within code called by the tests and have it listen to port 5005 as before.

    Files to read: When running the JUnit tests, besides the Surefire results in the target/surefire-reports folder the target/roller.log file provides logging of the temporary in-process TightBlog instance. Also, to determine any potential SQL/JPA problems, EclipseLink JPA logging can be activated, with the output file also in the target folder, by uncommenting the eclipselink.logging.* properties in the app/src/test/roller-custom.properties file.

  7. How to run Selenium tests - TightBlog's Selenium tests test very basic functionality (setting up a blog and saving a blog entry) but can sometimes serve as a quick sanity check that minor, last-second source code changes did not cripple the application. Navigating to the trunk/it-selenium folder and running mvn clean install will cause a temporary instance of TightBlog to activate that is subsequently used by Selenium to run its tests in Firefox. For these tests, none of the database or Tomcat configuration created above will be used, instead TightBlog will be running an embedded Jetty and a temporary (in-memory) Derby database, and shut down once Selenium is finished.

https://web-gmazza.rhcloud.com/blog/date/20151124 Tuesday November 24, 2015

TightBlog: November 2015 update

Since my update last month, I've been able to pull out an additional 35 Java source files, dropping TightBlog to 297 total compared to Roller's 493. The GitHub Issues List notes 114 issues closed and 21 still open. The application still isn't ready until the open issues are closed, but I'll keep working on zeroing out the remaining open issues.

The code simplification/shrinkage comes from normal refactoring and code modernization, embracing the Spring framework, using JAXB over JDOM, and having a higher threshold than Apache Roller of which functionality to retain. Generally speaking, any functionality that fewer than say 3% of bloggers would use gets tossed out, and functionality that perhaps up to 10% would use but would not be a deal-breaker if TightBlog didn't provide has also been removed. This threshold has dropped me to perhaps about 20-30 features that I can focus on making solid for bloggers. When there is too much seldom-used functionality to maintain, core functionality starts to suffer, causing people who might have liked the extended functionality to turn away from the product anyway due to its substandard core, a situation I'm trying to avoid with TightBlog.

https://web-gmazza.rhcloud.com/blog/date/20151115 Sunday November 15, 2015

Pink Slips at Disney. But First, Training Foreign Replacements.

Sold Out Book

Journalist Michelle Malkin and Center for Immigration Studies fellow John Miano have teamed together to offer a new book Sold Out. It offers a critique of the various foreign worker visas offered by the federal government and its consequences on American workers. I just got the book yesterday and am still in its early stages but would highly recommend it for American IT professionals wishing to learn more about the consequences of these programs or (for those already well aware) just wanting to thank and support the authors for raising publicity about this issue during an election year. Happy to note at the time of this writing that the book is #1 on Amazon's Government and Management bestseller list.


Valid HTML! Valid CSS!

This is a personal weblog, I do not speak for my employer.