The Cerberus Project

Chapter 1:

Work completed and subjects covered between May and August 2014.

Summer Term Progress

As a result of a summer co-operative opportunity at RIT, significant progress was made over the summer in pursuit of furthering the progress of the project. This progress can mostly be classified as refreshed learning, as well as hardware acquisition and provisioning. In addition, a significant portion of research was accomplished, which helped to cement the decisions for the chosen Operating System, as well as the provisioning of the units.

Acquisition

Through several recycling programs at RIT, in addition to the good will of several departments (primarily Research Computing) - a significant portion of excess hardware was obtained, free of charge under educational application only. Two primary units in the stack, denoted as two Primary Compute Nodes were acquired. These two units formed the spine of summer progress, as they allowed for learning and sandbox testing to occur in a professional enterprise deployment atmosphere. The units, both products from SuperMicro, pack AMD Opteron 2384 Processors (8 cores, single thread) and 128GB of RAM in a 64-bit deployment architecture.

In addition to physical hardware acquisition, a significant amount of software was acquired - most notably the idea to use SLURM to manage the cluster (SLURM will be elaborated and properly explained later in this document). Existing and working in a research environment that also actively manages a cluster computing environment was intensely valuable and aided the acquisition of many ideas, as well as hardware.

Research

Off the record research was a valuable look a wide variety of aspects of the computing world. Downtime over the summer was in abundance, serving as an excellent opportunity to conduct research into servers and their enterprise implementations, as well as software and learning the common skills of a cluster systems administrator.

This research allowed for a comprehensive look at current, industry favorite, operating systems in the Linux sphere, and allowed testing for each. With each of these came a look at specific package managers, compatibility and ease of use with kickstart implementation, as well as other dependency compatibility.

While the project was not officially started until the fall semester, advance research allowed for valuable research and consultation to take place, in addition to becoming with potential pitfalls and difficulties while allowing the scope of the project to be concretely determined.

UltraGrid and 4K

Work contained within the co-op was also relevant and helped build and maintain necessary skills, as well as introduction to new relevant material and ideas as well. One of the main focuses of the summer work was the application and development of UltraGrid - an open source application to stream and transport 4K uncompressed and compressed data streams over IP networks in a variety of ways. This was of express interest since 4K video files are relevant to this thesis project as well. One of the most valuable assets of working with UltraGrid was the attached networking skills that were gained, as well as focusing on the differences and advantages of CPU vs. GPU computation. UltraGrid focuses on its most efficient method of compression by taking advantage of NVidia's CUDA technology to compress high bandwidth streams into a MotionJPEG format. Working with this was excellent to compare against current DXT and other CPU intensive compression schemes.

Skills and Practices

The summer evaluation period also allowed for time to be dedicated to refreshing personal skills and memory of necessary components of the project. Most notably, these involved some skills that an advanced admin would find basic, yet are integral to the success of the project.

Networking

Networking was one of the most foremost aspects of computing that needed to be reviewed and the summer allowed for testing of networks as well as dealing with hosts, advanced manual networks and fighting with common services such as DHCP and DNS. RIT has an interesting network, especially when it comes to device registrations and getting familiar with all of the inner workings available to a student took some time as well.

Terminal + Shell

In addition to networking, familiarity with the linux terminal and the programming linux scripting language, known as Bash (or shell), also needed to be revived. Scripting in shell allows a great deal of system tasks to be automated, while also allowing complex programs and operations to be executed as well. Shell is the icebreaker back into traditional programming, helping to prepare the way for more advanced, upper level programming languages to be utilized later in the project.

Not every machine in the project utilizes a GUI, or graphic desktop interface, for the user to supply commands and interact with the machine. This can all be handled through Terminal, which comes native on all Linux machines, Mac OSX UNIX machines and Windows machines through the addition of third-party software such as Putty or Cygwin. Commands can be executed and each machine can be operated fully through the terminal, thus allowing a stack of servers to be configured in a 'headless' manner - in such a way that no keyboard, mouse or display is required for each machine. All of the machines can be controlled remotely, even from different rooms, buildings or cities, through a terminal instance once properly configured. This is a massive performance gain for using Linux machines, as once they are powered on, properly provisioned and connected and registered to the network - they never need to be hooked to a monitor or keyboard again, save for instances requiring individualized maintenance.

Project Management

Time over the summer also aided in getting some idea of project management in order. While a dedicated software suite for project management was left undecided until a given time in the Fall semester, the projects timeline was drawn up late in the summer. Generating timelines this early in the project allowed for all considerations to be made in a rough manner while still allowing flex blocks to be inserted in case of OS changes, or other project deviations without causing total destruction of the timeline and cause to build a new one. Goals, milestones and points of interest were possible to determine ahead of time once initial summer research was out of the way which provided this project with a clear advantage in a temporal sense over other projects which were left to preconceive and determine project management techniques during the Fall semester.