Outreachy: About the project

Posted on Tue 05 June 2018 • Tagged with outreachy, openstack

When it came to choose a project for which to apply in Outreachy, I was looking for 3 things: mentors are in about the same timezone as me, the project is in tech stack I'm familiar with and want to expand my skills in it, and last but but not least, the project's domain is something interesting and completely new to me.

With the last point I set myself up for a lot of reading (and some videos).

At first I felt like that dog in meme 'I have no idea what I'm doing'. To wrap my head around this, I started to draw a picture with some boxes. Here is the result:

diagram: Context of sushy

The project I'm working on is sushy and sushy-tools project.

sushy is related to, but not exactly part of, OpenStack Ironic project which deals with bare metal provisioning. Usually when talking about clouds, talk about virtual machines (VM), but there are cases where VMs do not provide necessary performance, so non-virtualized environment is necessary. Here comes Ironic project to manage bare metal servers in cloud environment - remotely. Ironic can be used independently or together with other OpenStack projects with whom it integrates. sushy is written in a way that it does not depend on Ironic and can be used by other projects. And Ironic can decide to use something else instead of sushy. But what does sushy do? Time to introduce Redfish.

Redfish is a standard API to work with bare metal servers. It lives in BMC (Baseboard Management Controller) which is a microcontroller (small computer) attached to motherboard of industrial servers. BMC allows to manage servers remotely and Redfish is one of the protocols to do it. The Redfish standard is managed by DMTF (Distributed Management Task Force).

sushy is a client library in Python for Redfish RESTful web services communicating in JSON. ironic imports sushy and uses it as one of the drivers. sushy is not the only Python library to consume Redfish API, there are alternatives named very similarly: python-redfish [5] python-redfish-library [6]

Besides sushy there is also project sushy-tools which contains emulators for testing sushy. Otherwise it is challenging for developers to test Redfish as real server with BMC and Redfish is necessary. There are 2 emulators: sushy-static which serves static JSON files provided by Redfish project. Mockups can be found at White Papers and Technical Notes section[4] looking for DSP2043. There is Redfish Mockup Creator[7] to generate mockup files from a real Redfish service. But this is little use to me as I don't have access to real Redfish service, but nice to know just in case.

Static mockup file emulator is OK for read only testing, but it does not help much when want to test actions where some changes are necessary. In this case there is sushy-emulator which uses libvirt driver connecting to virtual machine mimicking real server.

DMTF also provides similar emulators, both static mockup files[8] and dynamic[9]. I haven't tried these yet, but might try them out later. With all the alternatives available, it appears that each project takes different approach, so it is not like they are copies of each other and in the end there is choice.

Lastly, there are some acronyms that I've seen floating around in relation to Ironic that are not directly related to sushy, but I had to find out what they are and how they are related.

PXE (Preboot eXecution Environment) is way to boot up servers from network. Computers supporting PXE has NIC (network interface controller) that is up and listening to commands from network even when server itself is turned off.

IPMI (Intelligent Platform Management Interface) is a way how to manage and monitor servers remotely.

PXE and IPMI have been used together to deploy servers, but they are supposed to be replaced by newer technologies addressing some of their drawbacks - HTTP Boot and Redfish [10].

libvirt, already mentioned above, is API to manage virtualization, supporting wide range of hypervisors, including VirtualBox, VMWare, Hyper-V.

As always, in hindsight this all speaks for itself, but then again while writing this I discovered new places to go though I can avoid them now - this is just enough for sushy. It will be interesting to revisit this at the end of project and see what has changed in my point of view.

Next time I will write about first tasks I'm working on that should allow me to tell more about sushy and Redfish.


Outreachy: How I got started with OpenStack development

Posted on Sun 20 May 2018 • Tagged with outreachy, openstack

The first week has passed and in this post I will cover what I did to get my first contribution during application phase and some follow up reading afterwards. Here I will focus on general tools used in OpenStack development workflow. In next post I will write about the project I'm working on.

It was easy to start developing for OpenStack projects and almost everything I needed to know is well documented in one long page[1]. The page contains some links to follow for more detailed information. One of such pages that I find useful is about commit messages[2] that not only writes about messages themselves, but about splitting commits in smaller patches where split is reasonable and somewhat isolated rather than creating one large patch which is harder to review for everyone involved. Also commit messages should be self-contained and hold enough information to understand what's being done and why without accessing other systems like issue/bug tracker. This also says that commit message bodies should be wrapped at 72 characters. The main page[1] only mentions commit message headers that should be 50 characters.

Then there are specific OpenStack code style guidelines for Python[3] that also references the general Python style guide, PEP 8[4]. Here I'm relying on tox to catch any violations.

To write some code, I cloned the necessary projects, branched, and made some changes. There were some tools missing from my development environment that I installed as I went by, and I did not encounter any setup related issues when running projects.

Once the changes are done, the next step is to submit it for code review.

Gerrit is used for code review, it used to be written in Python, but was rewritten in Java. Wikipedia says[5] that it was done to make it easier to run on different Linux systems, but it does not give any reference to source and a quick search online did not help me to find any historical discussions, but this is not so important here, was just curious. More about Gerrit it is written at [6]. Gerrit distinguishes between core reviewers and regular reviewers, something that I have not seen in other tools where coders enforce it manually. In such cases, it is not like they try to bypass core reviewer requirement (as system does not enforce it), but it makes hard to identify open code reviews that are missing core reviewers, so it is nice that Gerrit has this built-in.

There is a handy tool git-review[7] to make work with Gerrit easier - to submit code review just run git review instead of following instructions in Gerrit's quick intro[6].

Once code review submitted Zuul[8] gets involved. Don't have to interact with this much, but Zuul will be the one who will tell if there are tests failing and PEP 8 guidelines violated. This should have been checked locally before submitting code review, but there are always more integrations to run and other issues that might not show up in local environment. As there are many patches submitted within the same time frame, Zuul will queue them and test queued patches together according to their place in queue to catch any conflicts between patches before they get merged to master. Then when code review is done and usually it requires 2 core code reviewers to approve, the Zuul will merge the changes to master.

That's all for now, if I encounter something new or something I misunderstood in relation to workflow and tools, will share it in further posts.


Hello Outreachy

Posted on Sun 29 April 2018 • Tagged with outreachy, openstack

Hello World, this is my first blog post for my journey as Outreachy intern at OpenStack "Extend Redfish hardware provisioning tooling for Python" project.