Reclaiming our Infrastructure With Observium

When you’re at a small, growing company, it’s easy to forget about the engine underneath the glitz and glam of all-star support, value-packed hosting services, and the Jim Groom cult of personality. Luckily for us (or for you), at Reclaim, we never forget. Maintaining our infrastructure to be as reliable, scalable, and secure as possible is one of our top priorities, and we recently got set up with the Observium monitoring system to get ahead of any infrastructure issues before they become customer-impacting.

Observium is a low-cost, SNMP-based, extensible, and (reasonably) easy-to-setup platform that helps us by automatically collecting data from our virtual infrastructure and making it digestible and actionable. For example, if we want to get some information about processor usage (a decent benchmark for how hard a system is working) let’s take a look at our processor reporting graph for our BYU server:

Screen Shot 2015-11-25 at 12.02.17 PM

The graph shows us some great info we can take action on (or not!). Right now, you can see that the processor utilization is fairly consistent for the past 48 hours, but if I were to zoom out on the graph over time (I can go as far back as I want), we might see the processor usage steadily increasing. At some point, the team will get an automatic alert saying “hey, your processor usage is over a certain amount,” which would signal to us that we might want to upgrade that server to accept more capacity, or dive deeper into what might be causing the increase if it’s not from additional users. Either way, the data is digestible and actionable.

Here’s another, more practical example. This morning, we got a processor utilization alert on one of our shared hosting servers. Large numbers of users did not all just sign up at the same time for new hosting, so Tim thought some suspicious activity might be occurring. Here’s what the Observium graph showed us:

Screen Shot 2015-11-25 at 12.12.29 PM

The “spike” at the end is clearly an outlier in comparison to the rest of the time the server is available. Sure enough, Tim looked at our security logs and a suspicious computer was attempting to connect to the server thousands of times. Tim blocked the suspicious IP address and the utilization immediately went back down, as shown in the graph. Again, all of this happened without any impact to the customer’s service – we got the alert and were able to take action before it got so bad that support tickets came in.

In helping us move things along a little quicker and making our alerts more actionable, I integrated the automated alert system with Slack, our collaboration/chat application. Instead of sending emails to a monitoring mailbox or someone’s inbox where they could easily be archived or ignored, the integration exists to alert the entire team at the same time so someone can take action quickly (I mean, that’s kind of the point of an alert, right?) In addition, as a general practice, it is really easy to get into “alert fatigue” in a monitoring system where you end up getting a ton of alerts for stuff that really doesn’t matter much – by customizing our alerts so they really only trigger when “the poo hits the fan,” we don’t fall into that trap.

If a team member wants to get a more general overview on the health of a server, they can log in to the Observium web interface.

Screen Shot 2015-11-25 at 12.24.27 PM

Observium doesn’t do some things I would really like to see, for example, as-is the alert suppression/escalation (getting someone to “acknowledge” an alert has triggered before it triggers again) features are a little light, and I would like to see the ability to turn system events (aka, a server rebooted) into alerts. I would also like to see some of the automatic discovery capabilities for Linux hosts improved or built-in to the system, but in order to overcome this, I wrote a script in bash that automatically makes all of the snmp/firewall/ssh key/configuration changes required on a target server and successfully discovers the device in Observium. If you’d like a copy of the script, please let me know in a comment and I’d be happy to share. Other than these minor quibbles, I’m very happy with the solution the Observium team has put together, and you cannot beat the functionality for the price.

Observium comes in two “flavors,” the “Community Edition” and the “Pro Edition,” and the “Pro” edition only costs £150 (about $225) per year, way, way cheaper than some of the other monitoring solutions available. If it’s something you’re interested in, you only get the alerting functionalities and capabilities from the Pro version, so I highly recommend going down that route.

This platform is and will be a work-in-progress with more features to come, including application monitoring, more advanced alerts, and hopefully some reclaim-specific customizations. We’ll keep tuning this Reclaim engine so you can go create amazing stuff!

Who Am I Here?

One of my favorite quotes from the classic 1987 b-movie horror Stepfather is when Jerry Blake (played brilliantly by Terry O’Quinn) picks up the phone, only to stop himself to ask that age-old question: “Who am I here?”

This is exactly how I felt when I got the following email:


I received an email signed by me, sent by Tim Owens, and from a company I co-founded. Who am I here?! In addition to setting up Middlebury Create and getting Whittier’s DigLibArts site up and running on their Domains server on Friday, I’ve been on a steady diet of moving various sites that were hosted on UMW Domains to Reclaim Hosting. I like this kind of work because it keeps me sharp with migrations using the restore a full backup in CPanel (a very useful tool!) as well as managing permissions after transfer, any database snafus, and more. It also provides the opportunity to go through the same sign-up experience as anyone using Reclaim. I have to say it is pretty elegant.

I got the above email after setting up a new hosting account for and migrating over the domains,,,, and It’s an email I know well because many folks who signup for Reclaim respond to it to say thanks or ask for some help. And they get a response in short order. But this time I was sending it to myself, a bit of selfdoogfooding on Reclaim Hosting. And remember, “I’m not just the Reclaim Hosting President, but I’m also a client!”

Reclaiming Community at BYU with Known

BYU’s Community Portal

We’ve been really lucky at Reclaim Hosting over the last year two years to have had the good fortune of working with the good folks at Emory University, University of Oklahoma, CSU Channel Islands, and Davidson College. These four schools have believed in what we’re trying to do from the start, and they’ve been as much partners and collaborators as clients. What’s so cool is that it has not stopped with them. The work we have been doing with Brigham Young University’s IT department over the last 6 months represents an exciting continuation of Reclaim’s partnership with universities to help co-create and implement a vision of what’s possible. What’s more, Reclaim’s not working alone on this one, we have finally been able to get serious and work hand-in-hand with Ben Werdmuller and Erin Jo Richey at Known to make these visions a reality. But I’m getting ahead of myself here, let me back up a bit…

Last Winter BYU’s CIO Kelly Flanagan and Enterprise Architect Phil Windley came to UMW to find out more about Domain of One’s Own. They have been working on a broader project to define a University API across their campus, and they saw personal domains and web hosting as a way of providing students spaces to manage and control their data in a space of their own. Kelly Flanagan has recently written a post, “Personal Domains, APIs, and Portfolios,” that captures the spirit of their project, and pushes to the very limits of decentralized data ownership viz-a-viz a personal API:

Imagine a world where other sites on the web don’t hold your personal data, but instead request access to the data they need through your Personal API. Perhaps you grant them access to only the portions they actually need and restrict them from others. They use the resources they’ve been authorized to access, perform the business functions you desire, return results, and their access is revoked.

This is a perfect marriage with BYU’s University API project, and represents about as vast and ambitious a vision for fundamentally rethinking Higher Ed IT I’ve yet to come across in my 10 years of instructional technology. But actually creating such a vision starts in small, concrete ways. When we were showing Kelly and Phil Domain of One’s Own community site, that became a focus for BYU. They were very interested in building that for their pilot, and they signed on for it over the summer. The issue was the community site Martha Burtis and Tim built almost two years ago is beginning to show its age and neglect (less is not always more). Trying to reproduce what was created as a prototype at UMW for schools working with Reclaim didn’t make much sense. It wasn’t easily abstracted out; Martha has been promoted up to grander things, and we needed to come to terms with the fact that we aren’t software developers. We can help imagine and create the infrastructure, but we need to start brining folks into the equation who specialize in developing specific applications like this—particularly if BYU’s community hub is going to support the Personal API vision.

So, in late September Tim and I traveled out to BYU to have just this conversation around the Personal API, and how a re-imagined community hub might be the first piece of that puzzle. The trip was a great success for Tim and I because we made some real headway on the initial vision of an API-driven community site. What we came away with was not only BYU’s continued support  on this project—they rule!—, but also the go-ahead to bring in Ben and Erin of Known to start building BYU’s community hub on top of Known.

What does this mean exactly? Well, that’s the fun part. Known already has an API built-in, so if Domains at BYU “beings with the Known” (to quote the great Adam Croom) the entire experience changes. For example, what would it mean if when you signed up for BYU Domains the first thing you saw was not CPanel, but an on-boarding process that begins with the option of integrating your various social media services through Known.

Reclaiming Community at BYU with Known
Connect your Social Media through Known’s Convoy

What’s more, once you have done that, you default to a page that is a quick and easy publishing space to send your various work to your social media accounts (the personal API at work already).

Reclaiming Community at BYU with Known
The Known Dashboard in BYU Domains

So, as you can see this post in the default page for my BYU Domain becomes a means of pushing to my WordPress blog, Facebook, Twitter, Tumblr, etc. All this being handled by Known’s Convoy, which does the work of connecting your domain with these various social media sites automatically. Now, by extension, there is no reason this can’t integrate with BYU’s University API. There could be buttons underneath the various post types that allow students to send their work to a particular class using something like Canvas’s API.

Reclaiming Community at BYU with Known
Privacy matters

What’s more, built into each post are various privacy settings: Public, Members only (limited to the BYU community), and Private. [When talking to Tony Hirst about this, he recommended a fourth type called “off-campus” which allows students to post to their social media sites from Known, but not have it aggregated to the community site. I like that idea.] But where is this community site you speak of? Well, built into every students dashboard is a Community link.

Reclaiming Community at BYU with Known
Toggle to view community posts

And when I click on that I can see what is happening around the BYU Community within your stream. Not unlike Tumblr. You’ll notice my domain is now showing me Kelly’s post about Domains.

Reclaiming Community at BYU with Known
Posts from community read in my Domain

The reader is a view of the community, but ultimately much more once we get targeted data via APIs. it can be a view of all the posts from one course, all the post from your Facebook community,  all the public posts to Facebook from the BYU domains community, same goes for Instagram, Twitter, Tumblr, etc. (hence why the “off campus” feature in privacy setting could be OUseful). A targeted, community based reader of sorts that integrates course and distributed social media based on tags and data. A vision of aggregation heretofore only dreamed of in our edtech philosophy.

Reclaiming Community at BYU with Known
BYU Community Site Directory

What some of those tags and features might be is loosely framed in the site directory feature of the BYU Community site Ben and Erin have built, but that is only limited by how BYU wants to use their University API to integrate with the Personal API.

Reclaiming Community at BYU with Known
Elastic search in BYU’s community hub.

You can also search across specific content types, but this could easily be extended to search across social media sites, departments, courses, etc.

The work Erin and Ben have done on the Community Hub for BYU is phenomenal, and this is just the beginning. BYU’s community hub provides a model that we can now abstract and provide to other interested schools, and partnering with Known on this may very well changed the way we imagine Domains not only as personal empowerment online, but also community engagement.

Now, having Known as the default interface for BYU Domains in no way rules out the ability for users to install other applications, subdomains, etc. We are currently thinking through what’s the best way to provide a quick and easy way to create subdomains or install WordPress, as well as a simple means of toggle to CPanel to work from there. That said, the switch could be a major one for communities used to working in CPanel, so we to think intently about what this means for the community at large, and what works and what doesn’t. But thanks to BYU’s willingness to experiment, we have a partner that is helping us work through these questions currently, and we may have answers to some of these questions as soon as January.

These are very exciting times for Reclaim Hosting. It’s cool to see the visions we’ve been courting as a community for years and years start to take shape in an actual application built upon the values of an open and independent web.

Setting Up S3 Storage for Omeka

Omeka continues to be a huge draw for a variety of students, faculty, and librarians using Reclaim Hosting. And the good folks at the Roy Rosenzweig Center for History and New Media have been champions of our service from the beginning, and that has made a huge difference for us. One of the issues that has come up regularly is storage for Omeka sites, which by design usually have large archives of documents, images, etc. We tend to keep our storage space for our Student and Faculty plans fairly low (2 GBs and 10 Gbs respectively) because we are trying to keep costs low, and the sales line of “unlimited” storage space for shared hosting is impractical for us. We recently introduced an Organization plan that has 100 GBs for just these instances because the need is there. That said, if you have a lot of resources you might be better off with a service like Amazon’s S3—the backup redundancy is insane and you can’t beat the price.

Screenshot 2015-11-10 23.30.05

Over 8 months ago Tim Owens figured out Omeka has the option for pushing all uploaded files to S3 built into their code. It’s just a matter of setting up an Amazon S3 bucket with the right permissions and adding the credentials to your Omeka’s config.ini file to get it running. I was intrigued by the process, but Tim had taken care of it so I knew it was theoretically possible—but never tried it. Yesterday, however, I had the opportunity to help a Reclaimer get this up and running for their Omeka install. With some help from Tim on a couple of details I missed, I got it figured out. The rest of this post will be a step-by-step for setting up S3 storage with a self-hosted Omeka site. Continue reading “Setting Up S3 Storage for Omeka”

7 Minute Support

Last month I wrote about Reclaim Hosting support, highlighting the fact in August and September we had an average of 8 minute response time to ticket. I also joked we needed to make it even quicker. Well, looking at our stats, over the last 90 days we responded to 1290 tickets with a  media response time of 7 minutes! #NOBODIES

7 Minute Support

There is no half-stepping at Reclaim Hosting.

Storage Upgrades and Plan Updates

We regularly monitor trends with the support tickets and feedback we get daily to look for future opportunities to make Reclaim Hosting even better. If one person is struggling with something we can help them, if many people are all struggling with something it’s a sign we need to fix the issue at a larger scale. Incremental improvements like this are a win/win situation because it solves problems for a lot of people and reduces the number of tickets that come in to us needing similar responses.

One question we’ve gotten regularly since day one revolved around the amount of storage we offer with our hosting accounts. When we launched in 2013 we had a single plan with 1GB of storage. At the time I’m sure it seemed crazy given the world we live in where every other hosts proudly promotes “Unlimited” storage (with 7 asterisks telling you they’ll take it away if you try to use it). Much like the myth of 24/7 support this is another area where other hosts overpromise and underdeliver. I made a point as we built out our infrastructure to offer reasonable plans and limits that would meet the majority of our community’s needs. Last year we added a Faculty plan at 5GB of storage and then 5 months later we doubled both plans to 2GB and 10GB respectively. As we have grown we’ve continued to build and give back.

There are, however, edge cases where groups or individuals just need more space for their stuff. We’ve helped folks setup connections to Amazon S3 and showed them tips on storing media off-site, but that also makes it difficult to marry with the message of “reclaiming your data”. With all that said we’re proud today to begin offering a third plan, the Organization Plan, with 100GB of space for $100/year (a little over $8/month). The plan, like all of our plans, includes unlimited hosting of addon domains, subdomains, databases, and we include a free domain registration or transfer as well. At a little over $8/month we think it’s a great deal for groups needing to setup exhibits and archives, users with high storage needs for archival work, or any other purpose. I’ll also point out that upgrading your plan is always a great way to support the work we are doing here! We continue to promote web hosting at reasonable cost and build slowly the right way and we’d love your continued support. Any existing customer can upgrade their account in the client area easily.

The other small change we’re making revolves around the Student plan. In the past we did not allow addon domains on the student plan. It was a great low cost option with limited storage meant for setting up a single site or a small handful using subdomains. Buying and hosting additional domains required the Faculty plan. This led to lots of confusion with users and frankly uncomfortable conversations on our part where it felt like we were “upselling” the user on additional upgrades after they had just bought a second domain with us. So we’re getting rid of the distinction. Our three plans now share all the common tools and features, and functionality. The only difference going forward will be the amount of storage space you need, making it dead simple to make informed decisions.

We love getting feedback and even more acting on it by growing and building as well as giving back to the community. We hope it’s making a difference and that you’ll continue to trust and support us for years to come as we continue to make this an awesome place to reclaim your digital identity.

Working Remotely a Month On

So It’s been more than a month that I’ve been working completely remotely for Reclaim Hosting. I’m still in the incredulous stages of the transition, and having moved to Italy makes it a bit more surreal. I feel like both a tourist and a resident at once, given how new the idea of living here feels (my recent Tweets provide a play-by-play), but also how embedded I am in Antonella’s community of friends and family. It hasn’t sucked just yet, but the fact that I have been constantly traveling for the last two months means I also haven’t had the opportunity to settle in. But given I have a clean slate for the foreseeable future, that is beginning to change. Continue reading “Working Remotely a Month On”