Research >> Public Cloud OpenStack for HPC Prototype & Microsoft Application Cloud Hosting

Public Cloud OpenStack for HPC Prototype & Microsoft Application Cloud Hosting


The project objectives for this effort was to determine the feasibility of OpenStack cloud hosting of Windows-based tools developed by HPCMP, the Integrated Hydrodynamics Design Environment (IHDE) and Leading Edge Architecture for Prototyping Systems (LEAPS) software packages, with the goal of remote access from Navy/Marine Corps Intranet (NMCI) systems.



There were two studies conducted on this project

  1. Task 1 - CREATE Study
  2. Task 2 - Microsoft Study


Task 1 Overview

Create Study

The High Performance Computing Modernization Program (HPCMP) is chartered by the Department of Defense (DoD) with supporting science and technology efforts, which includes maintaining high performance computing (HPC) systems for use by the acquisition engineering user community. However, the resource requirements of this community may exceed the capacity of existing HPCMP systems. In particular, the HPCMP may not have the time and resources available to respond to all requests from the community.
Cloud computing provides a potential solution for scenarios where the demands of the user community exceed the local HPCMP resources. Therefore, The HPCMP is in need of a comparator of DoD Supercomputing Resource Centers (DSRC) with commercial cloud systems, to assess the feasibility of using these types of clouds to provide capability for HPC workloads. The goal of this project is to design, prototype, and demonstrate an implementation of the public cloud in a private cloud configuration for comparison with existing DSRC resources for a workload of resources.

In this project, Nimbis Services, Inc. configured a small hardware testbed of compute servers (approximately 64 compute cores) so that it behaves as a traditional infrastructure-as-a-service (IaaS) cloud, where end-users can request compute resources by starting up virtual machines that run on the hardware. This is the model used by popular commercial clouds such as Amazon’s Elastic Compute Cloud (EC2) and the Rackspace Cloud powered by OpenStack.

Once the testbed was configured to behave as an IaaS cloud, computational engineering applications of interest to the HPCMP were deployed on to the testbed cloud, and tests were performed to evaluate whether the performance of these applications would be sufficient for users from the acquisition engineering user community.


Task 1 Outcome

Nimbis Services, Inc. has a small OpenStack testbed based on Cisco hardware in an Equinix IBX facility. This location has access to DREN and synergy with other DoD DIACAP customers. OpenStack is an open-source cloud operating system suitable for public and private cloud environments.

Nimbis measured the responsiveness of a graphics intensive interactive application, Capstone, using video analysis. The results showed that the response time was (just) below the 195ms threshold for interactive applications, albeit with a substantial reduction in the effective frame rate relative to local visualization.

Nimbis also performed MPI network scalability testing using Kestrel, a high-fidelity multi-physics analysis tool. Although the testbed is limited to 64 cores, it is sufficient for evaluating the effects of virtualization on the network interface and memory subsystems for this application. Nimbis compared the performance metrics against bare metal on the same system. The results showed that virtualization overhead was on the order of 6-9% for lower core counts, with unpredictable results above 32 cores due to the heterogeneity of the compute hardware in the Nimbis testbed. Sustained network bandwidth was observed of approximately 20MB/s at 64 cores, which is well below the capacity of the 10GbE network used for MPI-IO.

The Nimbis team has access to the HPCMP CREATE portal where the project binary code, test files, and installation and configuration documentation have been uploaded. The project was completed within budget and on schedule.



  • High Performance Technologies, Inc. (HPTI)
  • User Productivity Enhancement, Technology Transfer, and Training Program (PETTT)
  • High Performance Computing Modernization Program (HPCMP)


Public Cloud OpenStack for HPC Prototype

Brian Schott

Robert Graybill


Task 2 Overview

Microsoft Study

This project requires that the solutions considered must ultimately be Authority to Operate (ATO) certified, support dozens of users in an interactive mode via portal, provide adequate application execution performance, and support Common Access Card (CAC) users.

One of the requirements of the project was that the solution needed to be able to support users running on Navy/Marine Corps Intranet (NMCI) machines. In particular, users cannot install arbitrary client software on their machines, so any solution must be compatible with officially supported software.

For practical purposes, this placed the following constraints on any potential long-term solution: users can only connect with a remote desktop client that is already installed on their machine. This limits solutions to the Microsoft Remote Desktop Client that ships with the versions of Windows installed on the existing machines. Web-based solutions are a possibility, but they must work with Internet Explorer.

This project explored Cisco-based OpenStack private community cloud alternatives for the short and long term, developed implementation plans, hosted selected applications (IHDE, LEAPS), provided quick-turn demonstrations, and addressed security requirements.


Task 2 Outcome

This report outlined the technical approach for the Microsoft Application Cloud Hosting Phase 1 project. Nimbis Services, Inc. has a small OpenStack testbed based on Cisco hardware in an Equinix IBX facility. OpenStack is an open-source cloud operating system suitable for public and private cloud environments.

Nimbis deployed Windows 2012 virtual machines into their testbed, and deployed LEAPS and IHDE onto these virtual machine images.

Nimbis evaluated the feasibility of using remote GPU acceleration for improving the performance of interactive pre and post processing, and concluded that remote 3D acceleration was not possible given the constraints of the NMCI network. However, a subjective assessment by an end-user suggested that the unaccelerated performance was satisfactory and either comparable or superior to a previously used remote desktop solution.

Nimbis configured the virtual machines to support IHDE parallel processing via the Remote Execution System, but encountered problems with the batch files generated by IHDE. To date, these problems have not yet been resolved, although we are confident that these could be solved with sufficient time and with the assistance of someone with access to the source code.


Nimbis Services, Inc., is a trusted name in collaborative high performance computing. We help build communities for design, modeling, simulation, and analytics in the cloud. Learn more about Nimbis Services.