Glen Mazza's Weblog

Main | Next page »

https://web-gmazza.rhcloud.com/blog/date/20140929 Monday September 29, 2014

Activating Transport Layer Security (SSL) for web services

See how to implement SOAP web services over SSL.

[Read More]

https://web-gmazza.rhcloud.com/blog/date/20140922 Monday September 22, 2014

Creating a Java-first web service using CXF or Metro

Summary: This article shows the changes needed to the WSDL-first DoubleIt tutorial to convert it to a start-from-Java (no WSDL) process.

[Read More]

https://web-gmazza.rhcloud.com/blog/date/20140915 Monday September 15, 2014

Creating a SOAP client with either Apache CXF or GlassFish Metro

Summary: This tutorial shows how SOAP clients can be created using either Apache CXF or GlassFish Metro. We'll be creating a Maven-based client for the eBay Shopping Web Service, which provides a full range of search and retrieval capabilities for items being auctioned on the eBay site.

[Read More]

https://web-gmazza.rhcloud.com/blog/date/20140908 Monday September 08, 2014

Creating a WSDL-first web service with Apache CXF or GlassFish Metro

Summary: This tutorial shows how to create a WSDL-first web service using either Apache CXF or GlassFish Metro. We'll be using Maven as our build tool, with web service deployment on either Tomcat (for CXF or Metro) or OSGi deployment with Apache Karaf (CXF only).

[Read More]

https://web-gmazza.rhcloud.com/blog/date/20140614 Saturday June 14, 2014

Creating a Java Swing alternative to JConsole for calling MBeans

In this article I show how to create a Java Swing application to interact with MBeans in a manner specific for your needs, reducing need to use JConsole.

[Read More]

https://web-gmazza.rhcloud.com/blog/date/20140427 Sunday April 27, 2014

Working with Apache Roller source code

This entry explains the development environment and process I use for working on the Java-based Apache Roller blog server. I'm blogging this article to make it easier for those wishing to help with Roller development, also to assist Roller users wishing to modify Roller trunk code to better suit their needs (perhaps to bring in additional themes beyond those pre-packaged by Roller.) I use IntelliJ IDEA community edition on Ubuntu, but other IDEs and OSs are also used by other Roller team members. Some prerequisites for programming Roller:

ToolBefore proceeding, make sure you can...
JDK 1.7 or 1.8Activate "java" and "javac" from any folder in a command-prompt window
Apache MavenActivate "mvn" from any folder in a terminal window
Command-line Subversion clientRun "svn" from any folder in a terminal window
Standalone Tomcat 7 or 8Can start and view Tomcat from http://localhost:8080
MySQL or Apache Derby databaseCan activate mysql or ij (for Derby) from any folder in a command-prompt window.
Firefox Web BrowserOptional; used for running Selenium tests without needing to activate Tomcat
SQL Query tool such as SquirrelSQLOptional but recommended; used for querying the Roller database instance.

Trying to keep this blog of reasonable size, if you're new to any of the above tools their usage and setup are all well documented elsewhere on the 'Net. If you're already a Java developer and have GUI alternatives for the above command-line tools they should work fine as well (but be mindful of Ben Franklin's admonition about deviating from the terminal window for your work. :)

The instructions here are for Roller trunk (upcoming 5.1 version), which is much simpler, faster to build, and more modernized compared to the Roller release (5.0.x) versions; most all development is now occurring on the trunk.

  1. Check out Roller source and add any desired additional themes - Do an SVN checkout of Roller trunk and build it via the standard Maven mvn clean install command. If you wish to add any additional blog themes, place them in the src/main/webapp/themes folder prior to building Roller. Make sure you have a successful Roller build (including JUnit tests) before proceeding further.

  2. Prepare database and (optional) mail configuration - Follow the short Chapter 5 and Chapter 6, Sections 1-4 of the Roller Install Guide (ODT). Chapter 5 provides the few commands needed to configure an empty MySQL, PostgreSQL, or Derby database and Chapter 6 the configuration of the roller-custom.properties file, optional Mail configuration, and the required Mail and JDBC JARs that will be needed in Tomcat's lib folder. The test roller-config.properties file that I have in my Tomcat lib folder is as follows:

    # Any properties placed below overrides Roller default settings defined in
    # the roller.properties file embedded within the Roller WAR, and should be 
    # stored in the CATALINA_HOME/lib folder for Tomcat.
    
    installation.type=auto
    mediafiles.storage.dir=/myfiles/roller-inst/mediafiles
    search.index.dir=/myfiles/roller-inst/searchindex
    
    # Roller file and logging level
    log4j.appender.roller.File=${catalina.base}/logs/roller-tomcat.log
    log4j.logger.org.apache.roller=INFO
    
    #MySQL
    database.configurationType=jdbc
    database.jdbc.driverClass=com.mysql.jdbc.Driver
    database.jdbc.connectionURL=jdbc:mysql://localhost:3306/rollerdb?autoReconnect=true&useUnicode=true&characterEncoding=utf-8&mysqlEncoding=utf8
    database.jdbc.username=myusername
    database.jdbc.password=mypassword
    
    # EclipseLink debugging
    eclipselink.logging.file=/media/work1/underthehood/apache-tomcat-7.0.29/logs/eclipselink-tomcat.log
    eclipselink.logging.level=FINER
    
    # Mail config (See Roller Install Guide)
    mail.configurationType=jndi
    mail.jndi.name=mail/Session
    

    Before moving on to the next step, best to deploy the app/target/roller.war to your Tomcat webapps directory, start Tomcat and confirm you can run the application at http://localhost:8080/roller. This is an important one-time check to ensure your database, roller-custom.properties file, Tomcat, and Roller WAR are all properly configured, and once confirmed you should be in good shape for all subsequent coding and debugging. If any deployment problems, make sure you've reviewed Chapters 5 and 6 of the Install Guide, and if necessary contact the Roller User's List for assistance.

  3. Create a script to start development with everything needed - For efficient start-up I've created a hackroller.sh script that opens up all needed windows and applications at once instead of having me needing to do so manually each time I start development. A simplified version of my hackroller.sh is as follows and explained below:

    hackroller.sh:

    gnome-terminal --geometry=132x24 \
       --tab-with-profile=HasTitle --title "ROL Trunk" --working-directory="/work/opensource/roller-trunk" -e "bash -c \"svn update; exec bash\"" \
       --tab-with-profile=HasTitle --title "ROL Trunk2" --working-directory="/work/opensource/roller-trunk" -e "bash -c \"echo -Dmaven.surefire.debug; exec bash\"" \
       --tab-with-profile=HasTitle --title "Servlet Container" \
       --tab-with-profile=HasTitle --title "IntellijIDEA" -e "bash -c \"sh idea.sh; exec bash\"" \
       --tab-with-profile=HasTitle --title "GEdit" -e "gedit worklog.txt /home/gmazza/hackroller.sh $CATALINA_HOME/lib/roller-custom.properties $CATALINA_HOME/logs/catalina.out $CATALINA_HOME/logs/roller-tomcat.log `find $CATALINA_HOME/logs/localhost*.log` $CATALINA_HOME/logs/eclipselink-tomcat.log /work/opensource/roller-trunk/app/target/roller.log /work/opensource/roller-trunk/app/target/eclipselink.log " \
       --tab -e "dolphin --geometry=600x500+1+1 /work/opensource/roller-trunk $CATALINA_HOME" \
       --tab -e "firefox http://roller.apache.org/ https://issues.apache.org/jira/browse/ROL http://localhost:8080/roller http://svn.apache.org/viewvc/roller/ https://analysis.apache.org/dashboard/index/org.apache.roller:roller-project"
    #  --tab-with-profile=HasTitle --title "SquirrelSQL" --working-directory="/work/tools/squirrel-sql-3.5.2" -e "bash -c \"sh squirrel-sql.sh; exec bash\""
    

    The above script opens a multitab Console window which in turn opens a few separate application windows. In particular:

    • I open two terminal tabs pointing to the Roller trunk branch. This allows me to build Roller from one terminal while possibly reviewing my changes (using commands svn status and svn diff, for example) in the second window. I echo the -Dmaven.surefire.debug in the second tab just to quickly remind myself the Maven string to add during JUnit test debugging (covered below).
    • I have a separate "ServletContainer" tab open to my home folder for manual activation of servlet containers (Roller is tested on Tomcat, GlassFish, and JBoss) or other applications from the command line. I don't need this as often due to the copy.sh script (discussed below) I now use.
    • I use a tab for starting IntelliJ IDEA.
    • I open the multi-tab GEdit text editor with several files that are handy for me during development:
      • worklog.txt - A scratchpad file of TODO notes, commands, etc. that I keep for personal use between coding sessions.
      • hackroller.sh - The very script above, I keep handy in case I need to make adjustments or additions to it.
      • roller-custom.properties - The Roller configuration file discussed above and used by all Roller deployments to my standalone Tomcat. Changes I made here are activated on the next Tomcat restart.
      • catalina.out, roller-tomcat.log, localhost*.log, eclipselink-tomcat.log - These are logging files filled by Tomcat and/or Roller during running, usually very important for debugging as error logging usually gets written to one of these files. When you see terse "System" or similar errors reported by Roller in the web browser be sure to check these files for the error in detail.
      • roller.log, eclipselink.log - These files are under Roller's target directory and are populated during JUnit test running via the Maven test phase, as discussed below.
    • Dolphin - A multitab file browser that I have opened to the Roller trunk code and Tomcat folders for easy access to files.
    • Firefox - Here I pre-open tabs to the Roller website, JIRA task tracker and Sonar issues list, source code repository, and default location for Roller once it's deployed to Tomcat.
    • SquirrelSQL - Commented-out by default but used when I wish to query the Roller database to see the values that Roller is reading/writing. SquirrelSQL will need your database's JDBC JAR added to it as well as have the connection information you configured in roller-custom.properties.

    For scripts such as the above make sure there's no whitespace after the ending "\" on each line (error messages will pop up otherwise), and use a leading # (as shown above for SquirrelSQL) for actions you wish to disable by default. The $CATALINA_HOME specified above, as usual, is the base folder for your standalone Tomcat installation. In addition to the above configuration, you may wish to keep handy the Roller trunk documentation, kept in OpenOffice format in the Roller/docs folder or online.

  4. How to build and deploy to local Tomcat - After a successful mvn clean install from the Roller trunk folder, I run this copy.sh shell script that I keep in that folder:

    fuser -k 8080/tcp 8009/tcp 8005/tcp
    rm -r /media/work1/underthehood/apache-tomcat-7.0.29/webapps/roller
    cp ./app/target/roller.war $CATALINA_HOME/webapps
    rm $CATALINA_HOME/logs/*.log
    rm $CATALINA_HOME/logs/catalina.out
    $CATALINA_HOME/bin/startup.sh
    

    For rapid iteration when I don't need to run the tests each time, I simplify my build and deploy process to a single line: mvn clean install -Dmaven.test.skip ; sh copy.sh.

    The above file first kills the Tomcat instance (via fuser -k), deletes the previous expanded Roller webapp directory on the Tomcat instance and copies the latest created Roller WAR over, and clears all the logs before finally restarting Tomcat with the new WAR. The new Roller will be accessible at http://localhost:8080/roller again using the same $CATALINA_HOME/lib/roller-config.properties and database (i.e., all database-stored information including test blog data created will be immediately picked up by the new Roller WAR.)

  5. How to debug on local Tomcat - Learning to debug from your IDE any webapp running locally on standalone Tomcat is frequently vital when troubleshooting and thankfully simple to do. First, add to your Tomcat CATALINA_OPTS environment variable:

    export CATALINA_OPTS=$CATALINA_OPTS" -Xdebug -Xrunjdwp:transport=dt_socket,address=5005,server=y,suspend=n"
    

    Then from IntelliJ IDEA (or Eclipse), go to menu item Run -> Edit Configurations, add a Remote Configuration with a name of your choice and accepting the given defaults, and select OK. If Roller is already running on Tomcat, debugging can be activated at any time by setting breakpoints in the code that the Roller app would be activating, selecting menu item Run --> Debug {debug config name}, and then proceed with any needed code tracing.

  6. How to debug unit tests - To activate a specific Roller JUnit test, navigate to the {Roller Trunk}/app folder and run mvn test -Dtest=TestClassName#OptionalTestMethodName, as explained in the Maven Surefire Plugin documentation. To debug (code trace) the JUnit test within your IDE, add the -Dmaven.surefire.debug setting, set breakpoints in your IDE within code called by the tests and have it listen to port 5005 as before.

    Files to read: When running the JUnit tests, besides the Surefire results in the target/surefire-reports folder the target/roller.log file provides logging of the temporary in-process Roller instance. Also, to determine any potential SQL/JPA problems, EclipseLink JPA logging can be activated, with the output file also in the target folder, by uncommenting the eclipselink.logging.* properties in the app/src/test/roller-custom.properties file.

  7. How to run Selenium tests - Roller's Selenium tests test very basic functionality (setting up a blog and saving a blog entry) but can sometimes serve as a quick sanity check that minor, last-second source code changes did not cripple the application. Navigating to the trunk/it-selenium folder and running mvn clean install will cause a temporary instance of Roller to activate that is subsequently used by Selenium to run its tests in Firefox. For these tests, none of the database or Tomcat configuration created above will be used, instead Roller will be running an embedded Jetty and a temporary (in-memory) Derby database, and shut down once Selenium is finished.

Additional Notes:

https://web-gmazza.rhcloud.com/blog/date/20140203 Monday February 03, 2014

Creating Selenium tests for Java Web Applications

To reduce the amount of manual testing needed for the Java-based Apache Roller blog server, I added a Maven submodule that uses Selenium for automated in-browser testing. Presently only basic Roller functionality is being checked (create a user, create a blog, blog an entry, confirm the entry was saved), but I expect it to be filled out more over time. Its structure may be useful for other Java projects wishing to incorporate Selenium testing. The submodule POM relies on the Jetty Maven plugin to activate Roller, Brian Matthew's inmemdb-maven-plugin to activate an in-memory (i.e., no files created) Derby database instance, and finally the Maven Failsafe plugin to activate the Selenium tests, necessary as the tests run under Maven's integration-test phase.

To see Selenium in action, testing Roller (requires Firefox and Maven 3.0.5):

  1. Check out the Roller source using SVN:

    svn co http://svn.apache.org/repos/asf/roller/trunk roller_trunk
    
  2. Run mvn clean install from the roller_trunk (base) folder to build Roller and have it installed in your local Maven repository (from where it will be read by the Selenium tests). Building itself is quick (about two minutes on an average machine), however the initial download of Roller's dependencies, if not already in your Maven repo, could take some additional time.

  3. Navigate to the roller-trunk/it-selenium folder and run mvn clean install or mvn integration-test. Selenium will activate Firefox at Roller's home URL (http://www.localhost:8080/roller) and run its tests.

Some notes on creating Selenium-driven tests for web applications:

Using Selenium IDE to generate browser actions. Reviewing the nicely succinct documentation for Selenium IDE and Selenium WebDriver is a great way to get started. Selenium IDE is a Firefox plugin that records manual interaction with the application under testing ("AUT", using the documentation's terminology) into a script, which can then be activated from Selenium IDE to automatically re-run the same actions against the AUT. After adding testing assertions and verifications and confirming the script is moving through the AUT successfully, Selenium IDE can then be used to export the script as Java to incorporate into your WebDriver-backed Maven submodule. After becoming acquainted with the WebDriver Java API by working with a few exported files, you'll most probably find yourself able to code additional tests in Java directly without need for Selenium IDE.

Making adjustments to Selenium IDE scripts. Due to the manner in which Selenium IDE populates HTML form fields (perhaps by direct manipulation of the underlying HTML DOM document), certain mouse, key, and focus DOM events are not activated as they would be with manual data entry, resulting in necessary JavaScript not getting activated. For example, a submit button which would become enabled via JavaScript as a result of the data entry fields all being filled manually may remain disabled when Selenium IDE populates the form, making it unable to click that button and proceed. This occurred on one of Roller's registration screens -- the solution was to look for the DOM event in the JSP or generated HTML source which is needed to trigger the necessary JavaScript:

<tr>
...
    <td class="field"><s:password name="bean.passwordConfirm" size="20" maxlength="20" onkeyup="onChange()" /></td>
...
</tr>

<script type="text/javascript">
function onChange() {
    var disabled = true;
    var openIdConfig    = '<s:property value="openIdConfiguration" />';
    var ssoEnabled      = <s:property value="fromSso" />;
...

...and then add a fireEvent command within Selenium IDE prior to the command for clicking the Submit button:

Command:  fireEvent
Target:   id=register_bean_passwordConfirm
Value:    keyup

Note the Value above is keyup and not onkeyup; also, the target ID can be determined by having the browser display the HTML source for the page.

Exporting test cases (or a suite of test cases) into Java. Note that the Java exported cannot be reimported back into Selenium IDE for subsequent modification, although you can always save another copy of the test cases as HTML, load it into Selenium IDE for tweaking, and then do another export into Java. Also, exporting into Java is not strictly required (the Selenium Maven plugin used by Apache JSPWiki as shown here can work with Selenium IDE's default HTML), although I would not recommend HTML as you'll lose significant object-oriented coding advantages including code reuse.

The Selenium IDE File-->Export Test Case menu item provides three JUnit 4-based options:

  • RC - Uses the older Selenium 1 RC API.
  • WebDriver Backed - Uses Selenium 2's WebDriver to implement the Selenium 1 RC API. Good for transitioning from Selenium 1 to 2.
  • "pure" WebDriver - Uses Selenium 2's WebDriver API. I exported using this option, as presumably all new work should be based on Selenium 2.

In looking at the exported Java class(es), you may see commented "errors" about fireEvents (and possibly other commands) being unsupported, for example:

// ERROR: Caught exception [ERROR: Unsupported command [fireEvent | id=xxxx | keyup]]

This is usually not cause for alarm--the Selenium team decided not to support fireEvents in Selenium 2, feeling that WebDriver should instead internally fire the events that would occur if the data was entered manually. Alternatively, in certain cases testers can add actions that will cause those events to naturally activate. In my particular instance with the Roller submit button, it turned out no replacement coding was necessary as WebDriver, unlike Selenium IDE, was able to automatically fire the needed events based on the fields it filled. Note, worst case, it remains possible to execute JavaScript to fire the DOM events manually if the Java tests will not work otherwise, but before doing so, best to Google and/or search the Selenium Users Group with the specific "Unsupported command" message to see if a more standard solution is available.

Examine better ways to design tests. When working with the Java test classes, ways to improve their design using standard object-oriented techniques will become apparent. Foremost is moving to the Page Object design pattern (links). Thomas Sundberg's article shows the natural process of getting to that pattern by way of factoring out common functionality from the tests and additionally suggests using Cucumber for behavior-driven development. Some other suggestions:

  • Create an abstract base class for your page objects to handle common functionality--populating fields, validating screen titles, taking screen snapshots or logging the page source for errors, etc.

  • Although using the WebElement.click() method on form submits will normally halt processing until the next screen appears (and so far has always worked for me), the FluentWait object can also be used to explicitly halt Selenium until a specified HTML element on the new page appears (or a timeout you specify occurs).

  • For your page objects, create an additional multi-parameter constructor to allow for convenient creation of page objects in cases where the page is just being used to get to another page that is under testing. As such a page being activated with this constructor would not be under testing itself, just providing the minimum number of parameters in the constructor necessary to navigate to and test the desired page should be sufficient.

  • For time, accuracy, and efficiency, I would advise against turning your page objects into POJO's, with instance variables for each screen field and getters and setters for all fields. So far, I've added getters and setters for a field only when such a method is needed by a test case. Further, I'm not creating member variables in the page object for each widget, both to simplify the objects and out of fear that their values might deviate from what's actually on the browser screen. Instead, each accessor directly reads from or writes to the browser screen.

  • If you do wish to go the POJO route, take a look at the PageFactory object and @FindBy annotation, both described well on the ActivelyLazy blog.

  • In the Page Object model, when a submit button always moves the application from Screen A to Screen B, a typical Page Object method that your test classes will call will be as follows:

    public class LoginPage {
    ...
        public UserDashboardPage loginToApp() {
            // clickById() provided by AbstractRollerPage superclass
            clickById("login");  
            return new UserDashboardPage(driver);
        }
    ...
    }
    

    What do you do, however, if the subsequent screen could vary depending on the state of the application--for example, a login page might take you to a password-has-expired screen, a message notification screen if messages present, or the usual application screen if neither of the other cases hold? According to this article, it's recommended to have the page object implement different methods based on each different output possible, and have the test case call the appropriate method it's expecting based on the application state that it has created:

    public class LoginPage {
        ...other method above...
    
        public ChangePasswordPage loginToAppPasswordExpired() {
            clickById("login");
            return new ChangePasswordPage(driver);
        }
    
        public UserNotificationPage loginToAppUrgentNotification() {
            clickById("login");
            return new UserNotificationPage(driver);
        }
    ...
    }
    
  • Typically the constructor of a Page Object includes a sanity check verification that the page title is as expected (i.e., the WebDriver is actually on the page it is presumed to be on), throwing an exception if it is not. If you have multiple screens with the same title an alternative check based on an HTML element ID can be done. This will require that each page sharing the same title has a unique HTML ID attribute on an HTML element present only on that page, so you may need to have the page markup modified to include such an attribute.

Other Notes:

  1. To run multiple iterations of the same Selenium tests under different circumstances (e.g., using different security authentication methods), Juan Pablo of the Apache JSPWiki Team developed a WAR overlay method along with parameter filtering to configure each of the tests - check the JSPWiki IT Tests module to see the process.

  2. Functional Automated Testing Best Practices with Selenium WebDriver - presentation by Ben Burton

https://web-gmazza.rhcloud.com/blog/date/20140123 Thursday January 23, 2014

Advisement on traveling to Montreal/Toronto via New York with Greyhound Bus

I traveled to Montreal this past weekend via Greyhound from Washington DC (photos). For added flexibility, although it's a bit more expensive, I wanted to buy two one-way tickets instead of a round-trip, with the second purchased while I was in Canada, on the day I'm planning on leaving. This way, if there's a snowstorm in Canada I can leave a day later or earlier as necessary to escape blocked roads, or I can leave earlier or later in the day depending on my mood, etc., giving me much more flexibility. Buying separate tickets has not been a problem for me in the past, as three of my past four trips to Canada (one via MegaBus to Toronto and two via flight to Ottawa) I didn't buy a return ticket until I was in Canada.

Travelling to Montreal, the Washington to New York segment was fine but Greyhound refused to allow me to board the New York-to-Montreal segment because I didn't have a pre-bought return ticket. According to the lady taking my ticket at New York, validated by two other employees there, U.S. passport holders must show a pre-bought return ticket prior to being allowed to board. I was told this in stunned disbelief at 11:48pm of a 12:01am bus trip, for which the next available bus would be at 8:30am the following morning. After scrambling around the bus terminal trying to find where in the huge station I could buy Greyhound tickets, I was able to buy a return ticket just after midnight at the station, and then rush back to the gate. Thankfully, due to bus delays I was able to board and continue my trip, but this was a traumatizing 20-25 minutes for me that could easily have become an 8.5 hour wait in the bus terminal for the next bus if I couldn't buy a return ticket in time. Further, as my ticket to Montreal was print-at-home, it could have been declared invalid for the 8:30am ride, requiring me to buy yet another ticket north--however the Greyhound employee told me they probably would honor my 12:01am ticket for a later, same-day ride.

There is apparently an exception for U.S. passport holders with a visa indicating that one can stay forever in Canada, but that might be dependent on the Greyhound employee taking your ticket. Also, the lady taking my ticket had indicated the return ticket of a plane, train, or any company's bus ticket would be sufficient. Further, for simplicity, I could have bought the ticket right there on a smartphone, and shown it to her that way. If planning on driving back from Canada, one Greyhound ticket employee recommended buying a refundable ticket for the return. But, again, the precise terms may vary based on the individual Greyhound agent taking your ticket. Indeed, speaking with the Greyhound ticket agent at the intermediate Albany, NY station, he indicated his station at least does not have a round-trip purchase requirement at all. There seems to be too much variance on precise boarding requirements, and the Greyhound website does not mention the need to have a pre-bought return ticket, so I just wanted to blog my experience to help others avoid my experience. As two one-way tickets, one station-bought and hence more expensive, ended up costing much more than a round-trip ticket ($196 vs. $135) consider getting the latter to save the hassle and expense of a last-minute station purchase. Also good to confirm your tickets and passport with a Greyhound employee once you get to New York as soon as possible to confirm that you'll have no last-second problems boarding.

I felt Greyhound should have clearly stated in its "Traveling to Canada" section on its website that travelers with U.S. passports going through NYC must have a return ticket prior to being allowed to board a bus going to Canada. Had I known that, I would have happily bought the round-trip to begin with. I called Greyhound customer service and spoke with two levels of support there, but ultimately could not get a $60 refund of my additional costs incurred, as they felt the text in that section was already clear enough about the need to produce a pre-bought return ticket prior to boarding and they don't have a process for partial refunds of the type I was asking anyway. However, the 2nd level help said she might put in a suggestion to have the travel requirements more clearly stated on the website. Until then, incoming Googlers have my blog entry here. :)

Having all non-Canadian passengers possess a return ticket is not necessarily a bad thing--a bus cannot continue past the border until all on the bus are cleared, or those unclearable taken off the bus and sent to the adjacent U.S. customs side. Having a return ticket helps confirm to Canadian customs that you will be leaving their country, hence reducing the number of people sent to secondary and helping speed up the trip for all on the bus. Also, possession of a return ticket prevents disasters of people being refused entry and stuck at the Canadian border, 360 miles north of NYC, as it guarantees they will have a return ticket that they can use to get on a Greyhound going back home.

Outside of this hiccup, I must say, the drivers drove safely and efficiently, both going there and coming back, the buses were cozy and some of them were downright luxurious (leather seats!) and the WiFi worked well. Arguably, they did such a good job they deserved the extra $60. Other notes: for web purchases, tickets picked up via the Will Call option are generally much easier to exchange than the print-at-home ones possibly due higher chance of fraud the latter present, and Greyhound has tightened up its ticket exchange rules so make sure you well understand the ticket exchange policy on its website prior to buying any tickets when you are unsure about when you really want to go and come back. Also, although secondary inspections are infrequent, expect your belongings including laptops to be fully searched when you cross the border in either direction, so don't carry anything of an embarrassing or inappropriate nature.

https://web-gmazza.rhcloud.com/blog/date/20131216 Monday December 16, 2013

Hosting an Apache Roller blog on Red Hat OpenShift

Updated January 2014.

Prefer a Wiki instead? See my JSPWiki on OpenShift entry.

This tutorial outlines the steps to deploy Apache Roller 5.1.0-SNAPSHOT (unreleased) on Red Hat's OpenShift public PaaS, the deployment used by this blog. I'm using the unreleased Roller version as it's been much simplified compared to earlier versions and almost all current development efforts are on this version, however, the latest release (5.0.3)--which may have fewer kinks--can also be used. I'll be deploying on JBossEWS 2.0 (Red Hat's commercial offering of Apache Tomcat 7.x and the Apache web server) using MySQL database and a Google Mail account to handle email notifications.

OpenShift is free within certain usage bounds, in particular three free "small gears" are provided. I'm deploying JBossEWS as scalable, meaning that it will take up more gears (up to the free maximum) as demands increase; however one of the three gears presently needs to be consumed by MySQL. (At a later date I hope to look into using Apache Derby for Roller instead of MySQL. With Derby on the JBossEWS gear(s) the MySQL installation can be deleted, freeing up a gear.) Note OpenShift also provides PHP-based WordPress as a blogging option; you may find Roller preferable if you're a Java developer and/or already have a JBoss or JBossEWS instance running on OpenShift to host your blog.

  1. Customize and build an Apache Roller war. Check out and build Apache Roller from trunk, i.e., 5.1-SNAPSHOT. Prior to building you may wish to add additional blog design (theme) options for your blog server--this blog uses the Rolling theme from the Roller-extras project. Next, you may wish to rename the roller.war generated to create a different URL path for your Roller blog (all blogs besides the default blog will have the blog handle added to that suffix). If you can, making Roller the default application by naming it ROOT.war (and deleting or renaming the already existing root application in the Tomcat webapps folder) will result in simpler URLs for permalinks--this is the approach I've taken for this blog. The following shows a URL comparison of a blog.war having a blog with handle of "gmazza" versus a ROOT.war with a blog handle of "blog":

    blog.war with blog handle of "gmazza":

    blog home:      http://web-gmazza.rhcloud.com/blog
    specific date:  http://web-gmazza.rhcloud.com/blog/gmazza/date/20131216
    specific entry: http://web-gmazza.rhcloud.com/blog/gmazza/entry/apache-roller-on-openshift
    

    ROOT.war with blog handle of "blog":

    blog home:      http://web-gmazza.rhcloud.com (also http://web-gmazza.rhcloud.com/blog)
    specific date:  http://web-gmazza.rhcloud.com/blog/date/20131216
    specific entry: http://web-gmazza.rhcloud.com/blog/entry/apache-roller-on-openshift
    

    If you'd like your blog to use SSL, either the entire site or just pages in which sensitive information will be transferred, uncomment and adjust the <security-constraint/> element in the Roller WAR's web.xml file as appropriate, following the comments above that element. Further, when you get to Step #4 below, update your OpenShift project's server.xml and context.xml files exactly as described in the "For Tomcat (JBoss EWS)" section of this OpenShift knowledge base article.

    At this stage, you may wish to do a test deploy of Roller on Tomcat running on localhost, following Chapters 5 and 6 of the latest Roller Install Guide (OpenOffice format) to confirm your WAR and desired system settings are working properly; most of those settings will carry over to your OpenShift installation, making the latter installation smoother.

  2. Create a new JBossEWS 2.0 application. Sign up for a free OpenShift account if you haven't already. From the applications page, choose Add Application and select the latest JBossEWS option. You'll probably want to choose the scaling option so JBossEWS can scale to additional gears if needs arise (scaling cannot be later added in if you don't choose this option when creating the application.) The name you choose becomes part of the subsequent URL for the application, for me I chose "web" giving me a base URL of web-gmazza.rhcloud.com, which can be replaced by purchasing a domain name. I'm avoiding an application name specific to what I'm deploying, to make the URLs sensible should I choose to host multiple web applications (.../blog, .../wiki, etc.) on the same JBossEWS instance.

    After creating a JBossEWS 2.0 instance, make sure you can accomplish the following three tasks before proceeding:

    • ssh into your instance where you can run standard *nix commands to maintain your application (checking Tomcat logs and configuring some files). For the SSH connection string, go to the OpenShift application page and click on the "Want to log into your application?" question.
    • Clone your application's repository to your local machine. Much of the Roller configuration will be placed into this repository and pushed to your OpenShift instance.
    • Install the rhc client tools and run rhc setup. These commands allow for remote administration of your OpenShift instance.

  3. Add MySQL to your JBossEWS application. From the OpenShift page for your application, select "Add MySQL 5.1" or from your local machine, enter the command rhc cartridge-add mysql-5.1 --app web substituting "web" with the name you chose for your application. This command will create a MySQL database hardcoded to the name of the application; however, if you're using JNDI, Apache Roller is configured by default to look for a database called "rollerdb"; also, as explained in Chapter 5 of the Roller Install Guide the MySQL database needs to be configured to use UTF-8. So, we'll delete this database and create one that Roller can use. To do this, ssh into the OpenShift server and run the following commands--change the username and password below to something more secure:

    [web-gmazza.rhcloud.com ...long string....]\>mysql
    mysql> drop database web;
    mysql> create database rollerdb DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    mysql> grant all on rollerdb.* to scott identified by 'tiger';
    mysql> quit;
    mysql> set wait_timeout=90000;
    [web-gmazza.rhcloud.com ...long string....]\>exit
    

    Once Roller is installed and running you'll have easy access to the Roller database tables, just run mysql as above, enter use rollerdb; and enter whatever SQL statements desired, followed by a commit; for data changes. For example, I changed this blog's handle from "gmazza" to "blog" by updating the handle column in the website table, its corresponding objectid in the roller_permission table, and the site.frontpage.weblog.handle property in the roller_properties table (as this is the default blog). Then a commit followed by an app restart, and the new handle was activated.

    Besides using mysql at the ssh command-line to run queries, MySQL can also be accessed from your local machine, see Grant Shipley's article and video for some options.

  4. Create JBossEWS- and Roller-specific configuration files. Prior to deploying the Roller WAR, here are the configuration steps I followed:

    1. Move to the web/.openshift/config folder of the local copy of your application repository (obtained via git clone described in Step #3 above). Edit the file catalina.properties, adding the following new folders to the common.loader property:

      ${catalina.home}/../app-root/data/rollerdata/lib,${catalina.home}/../app-root/data/rollerdata/lib/*.jar

      The above setting is needed because the Tomcat lib folder where we would otherwise be placing configuration files and JARs is not accessible to the user on OpenShift JBossEWS, so we need to list additional folders for JBossEWS' classloader to look through.

    2. In the server.xml file in the same folder, add to the <Host/> element Roller configuration similar to the below, adjusted for your setup:

      <Context
        path="/blog"
        docBase="blog"
        antiResourceLocking="false">
        <Resource name="mail/Session"
          auth="Container"
          type="javax.mail.Session"
      
          mail.transport.protocol="smtp"
          mail.smtp.host="smtp.gmail.com"
          mail.smtp.port="465"
          mail.smtp.auth="true"
          mail.smtp.user="blah.blah@gmail.com"
          password="...email password..."
          mail.smtp.starttls.enable="true"
          mail.smtp.socketFactory.class="javax.net.ssl.SSLSocketFactory"
          mail.smtp.socketFactory.port="465"
          mail.smtp.socketFactory.fallback="false"
          mail.debug="true"
        />
      </Context>
      

      Update path and docBase above to what your Roller WAR is named. If you've made Roller the default application, i.e. ROOT.war, use "" and "ROOT", respectively, for these two values.

      The Google Mail configuration above is optional and primarily for for sending email notifications to the blogger when someone leaves a blog entry comment, either just as an FYI or for moderation before the comment is published. Probably best to create a new email account for this purpose rather than relying on a personal email address, as the server configuration requires a hardcoded email username and password potentially viewable by others. Using mail services other than GMail should also work fine. To activate emails, see Section 6.2 of the Roller Install Guide for servlet container configuration and Sections 7.1.3 and 9.2 of the Roller User's Guide for Roller configuration. This step is optional not only if you're not intending to use the emailing feature but because OpenShift will send the notification emails using its own "no-reply@rhccloud.com" account if you don't do your own email configuration.

    3. ssh into your application and create a rollerdata/lib folder structure under app-root/data. In this lib folder add the JARs described in Section 6.3 of the Roller install guide, namely the MySQL JDBC Driver and JavaMail's javax.mail-1.5.0.jar (the latter, required whether or not you'll be using emailing features). SCP or WinSCP provides an easy way to transfer files from your local machine to your OpenShift instance:

      scp mysql*.jar {appUUID}@web-gmazza.rhcloud.com:app-root/data/rollerdata/lib 
      

      Where the URL string above will need modification based on your application URL and {appUUID} is your Application UUID, obtainable from your local machine by running the rhc apps command or via whoami when logged into the OpenShift server.

    4. In the same lib folder we'll need to add the roller-custom.properties files containing our Roller configuration overrides (doable also via the SCP command above or editing a file after SSH'ing into OpenShift), similar to:

      installation.type=auto
      mediafiles.storage.dir=/var/lib/openshift/{appUUID}/app-root/data/rollerdata/mediafiles
      search.index.dir=/var/lib/openshift/{appUUID}/app-root/data/rollerdata/searchindex
      log4j.appender.roller.File=/var/lib/openshift/{appUUID}/jbossews/logs/roller.log
      log4j.logger.org.apache.roller=INFO
      
      #MySQL
      database.configurationType=jdbc
      database.jdbc.driverClass=com.mysql.jdbc.Driver
      database.jdbc.connectionURL=jdbc:mysql://{host:port}/rollerdb?autoReconnect=true&useUnicode=true&characterEncoding=utf-8&mysqlEncoding=utf8
      database.jdbc.username=????
      database.jdbc.password=????
      
      #Mail
      mail.configurationType=jndi
      mail.jndi.name=mail/Session
      

      Where:

      • {appUUID} is defined as earlier.
      • {host:port} is obtainable by running "echo $OPENSHIFT_MYSQL_DB_HOST:$OPENSHIFT_MYSQL_DB_PORT" while logged into the OpenShift server.
      • the database username and password is for the user granted access to the rollerdb database (the "scott" and "tiger" listed above), not the MySQL admin username and password.
  5. Upload the Apache Roller WAR to your OpenShift JBossEWS instance. Best to use the SCP method to upload the Roller WAR instead of uploading it via the Git repository. That's because the git repository storing your changes count against your gear's 1GB space limitation, and binary WAR updates in Git will quickly consume that space.

    Important: By default, OpenShift's haproxy bombards the JBossEWS root application with GET requests every two seconds to confirm the JBossEWS instance is running well. If you've decided to host Roller as the default application (ROOT.war), it will receive that bombardment, resulting in your blog reporting 1000's of hits each day and frequently filling up the Catalina error logs with "broken pipe" messages as Roller usually can't handle requests that rapid and continuous. To fix this, increase the pinging interval, disable the pinging entirely, or have it ping another webapp by altering the option httpchk setting in the haproxy.cfg file in your OpenShift instance. For my Tomcat instance I decreased the pinging to once per minute and had it ping an empty webapp: option httpchk GET /scalechk/, with a new jbossews/webapps/scalechk "application" containing only a static index.html file.

    Finally, do a git add/commit/push to place the configuration files in your local web/.openshift folder to the JBossEWS instance. (Important: You will always need to do a git push after making changes in your .openshift folder to propagate them to your OpenShift application.) The action hooks defined in your project will automatically shutdown and restart the Roller application as part of the git push. If all goes well you should now be able to access your Roller installation at http://{appname}-{accountname}.rhcloud.com/{war name} and continue the standard Roller automated setup. Roller will create the database tables for you and guide you into creating your first blog, etc., as described in the Roller Install Guide. Before continuing, make sure you can create a blog, a blog entry, and (if you've configured Roller to do so) that you're receiving email notifications for new comments. The Roller User's Mailing List is available should you have any configuration difficulties.

  6. Perform post-installation configuration. Once everything appears to be running fine it would be good to reduce system logging and remove unneeded webapps in order to limit server usage and speed Roller performance. Some suggestions:

    1. For safety and performance you will probably want to remove any pre-installed applications under webapps/ in your JBossEWS installation.

    2. Reduce Tomcat logging: Logging configuration is done in {appname}/.openshift/config/logging.properties of your Git application repository. Changing the Catalina logging from FINE to WARNING will considerably reduce the amount of logging stored in the catalina.out and catalina.{date}.log files:

      1catalina.org.apache.juli.FileHandler.level = WARNING  
      java.util.logging.ConsoleHandler.level = WARNING
      

      Other worthwhile suggestions include deactivating the catalina.out and any unused loggers (e.g., the manager and host-manager loggers if you don't have their corresponding applications installed.)

    3. Shut off the Tomcat AccessLogValve: this logs whenever a GET or a POST is made against Tomcat, it can be deactivated by commenting it out within the .openshift/config/server.xml file:

      <!--Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
          prefix="localhost_access_log." suffix=".txt"
          pattern="%h %l %u %t "%r" %s %b" /-->
      
    4. Reduce the Roller logging down to WARN level: Roller is extremely chatty at the INFO level, setting its logging level to WARN (not WARNING) in the roller-custom.properties is preferable for post-installation use:

      log4j.logger.org.apache.roller=WARN
      

Notes/Additional Resources:

  • Sometimes the git push origin master command fails to reactivate the MySQL database even as it will still activate JBossEWS. You will be able to see by logging into your account from OpenShift website and checking your application's status page (https://openshift.redhat.com/app/console/application/{appID}). If this occurs a manual rhc app stop {appname}/rhc app start {appname} from your local machine should fix the problem.
  • Better Blogging with Apache Roller - Learn about blog backups, setting up Google Analytics, blog widgets, etc.
  • OpenShift documentation
  • The OpenShift Forum is available for community-based help.
  • If the web application becomes inoperative and you can't ssh into your OpenShift account, it may be an out of memory or threads issue. Best to run the rhc app stop appName command (or, if necessary, the stronger rhc app force-stop appName) from your local machine followed by another ssh attempt where you can then work to fix the problem with your application.
  • See this note for using a more friendly domain name besides the standard appname-accountname.rhcloud.com.

https://web-gmazza.rhcloud.com/blog/date/20130617 Monday June 17, 2013

Better Blogging with Apache Roller

Tips and tricks on blogging with Apache Roller.

[Read More]


Valid HTML! Valid CSS!

This is a personal weblog, I do not speak for my employer.