Azuracast: One-click Web Radio in the Reclaim Cloud

Yesterday was a win because I finally focused for long enough to work through creating a one-click installer for the open source web radio application Azuracast. I’ve had a couple of conversations with folks around web radio this week, and I have to say it was cool to hear them suggest web radio is one area they want to explore. I love the whole idea, and I figured Reclaim Hosting could do its small part to make installing and hosting Azuracast that much easier. I have played with the software a bit in Reclaim Cloud to create Reclaim Radio, stand-up Strawboss Radio for the inimitable @scottlo, and finally to migrate #ds106radio to Azuracast. So I had installed the software using their Docker container a few times now.

The trick was to take the installation commands and create a one-click installer for Reclaim Cloud. Tim has already created quite a few, and he wrote-up a bit about the approach for creating packages on the Reclaim Community forums that was a very useful starting point:

Jelastic has documentation at https://docs.cloudscripting.com/ on how to develop these packages which take the form of a yaml or json file. You can also browse all of the packages currently available via Jelastic’s Github organization at Jelastic JPS Collection as well as recent additions I’ve built and added to the Marketplace at Reclaim Hosting’s Github. The repos contain a manifest.yaml file which has all of the necessary code. In more complex scripts that file may call other scripts within the repo to do various things.

One method I’ve had a lot of success with is creating a generic Docker container and then running a build script to automate pulling down and running the particular software project.

Tim points to the example of the manifest file for RStudio he created as a model, which is where I started. I created my first Github repo in the Reclaim Hosting account (which is a proud moment for me 🙂 ) and I got started. And, in fact, it was pretty simple, I had to update the application name and other details, but the only real significant change to the RStudio manifest he shared was to the actions section:

actions:
  setup:
    cmd[cp]: |-
      mkdir -p /var/azuracast
      cd /var/azuracast
      curl -fsSL https://raw.githubusercontent.com/AzuraCast/AzuraCast/master/docker.sh > docker.sh
      chmod a+x docker.sh
      yes no |./docker.sh install
      #Checking of container is up and running
      until [ "`/usr/bin/docker inspect -f {{.State.Running}} azuracast_web`"=="true" ]; do
        sleep 1
        print '.'
      done;
      echo OK
ssl: true
skipNodeEmails: true
success: |
  **Azuracast URL**: [https://${env.domain}/](https://${env.domain}/)

This is the finished product, but to be clear it took some finessing of my attempt by Tim to get it to work. The big thing was that the Docker container asks questions about the custom domain and SSL certification to creating and environment file, so that had to be forced in the script with the line yes no |./docker.sh install. The other thing is the test to make sure the container is running until [ "`/usr/bin/docker inspect -f {{.State.Running}} azuracast_web`"=="true" ]; do needs the name of the application for that Docker instance, which is azuracast_web, so that was another thing Tim helped me figure out.

That said, I can see how creating these one-click installer can be pretty do-able with the right container, which is pretty awesome. Even better, it is not necessarily limited to us creating these manifests, as Tony Hirst demonstrated with his Jupyter Notebook work, anyone can do it and share appropriating for others to use, where or not it is in our marketplace. So, all that said, I finally have my first attempt at an installer for Reclaim Cloud under my belt.

A Follow-up on Costs in the Cloud

Some new Reclaim Cloud art from Bryan Mathers, isn’t it beautiful?

Yesterday marks the first full month of my being a paying customer in Reclaim Cloud. I wanted to take the occasion to follow-up on my post in July where I started projecting what the costs of hosting on Reclaim Cloud might look like given you only pay for what you use, and that will be variable over time. The uncertainty of variable pricing can be scary, but at the same time once you have a sense of what to expect it starts making a lot more sense. So, below is a look at my total monthly cost for August hosting this blog, ds106.us, my own Jitsi instance (which I turn on and off as needed), Etherpad, a YouTube Downloader app, as well as a number of parked applications I’m experimenting with.

And the verdict is in, I spent $87.42 in August, and that breaks down as follows:

  • bavatuesdays (WordPress): $35.73
  • ds106.us (WordPress): $26.70
  • ds106club (Ubuntu VPS): $5.93
  • bavanotes (Etherpad): $5.77
  • bavameet (Jitsi): $4.23
  • Youtube Downloader: $2.89

These are the apps I regularly used, and that works out to $81.25, the other $6.17 was spent on the Ghost ($3.04) and Discourse instance ($3.04) the former of which I eventually shutdown and archived to a static site. The remaining .09 cents I spent on testing various environments with a few different apps.

The nice thing about the Cloud billing is it gives you insight into exactly how many resources you used and what they cost at the container level and beyond. Above I exploded the view for this blog and you can see the costs are split up given I moved the environment mid-month. The costs are predominantly for the LEMP application server ($31.37) and the rest is for the dedicated IP address ($3) and the CDN I installed ($1.36) brings the total to $35.73. This is about half what I would be paying with Kinsta, which is nice to know, and about $5 more than Digital Ocean, although that did not include a CDN.

The big win for me once Tim added the Edgeport CDN service that Reclaim Cloud is utilizing is that I really had no more reason to pay Cloudflare for their CDN and caching service, effectively saving me $20 per month (actually $40 if you include I also had a pro plan for ds106). So, I am effectively $90 monthly for my web services through Reclaim Cloud, with a fair bit of room to experiment. I could probably shave off $8 or $9 if I started and stopped Etherpad and the Youtube Downloader application. I think this kind of monitoring of usage makes me consider not only what I am spending, but what resources I am using. And this is a bit different then just minding money—I could probably skip the bill and no one would say anything 🙂 —it’s the fact it makes me more mindful of what I can shutdown to save resources—such as Discourse and Ghost—it goes back to the idea of turning off the lights when you are not in the room. Same thing here, and it is what is unique about the Cloud, in shared hosting I spend far less, but at the same time I hog resources with the 15-20 applications I have installed (maybe two of which I am actively using) and never think twice about it.* I’m not sure sustainability is the right word given its environmental connotations, although it might be, but it does make sense in terms of ensuring our shared hosting infrastructure is not overburdened with accounts that demand more resources, such as mine.

Anyway, I guess I am still blogging about this because I’m so used to paying $30-$100 a year to host any and everything I want on a shared hosting server and I am trying to come to terms with a digital life outside the “free” third party trap services is something I have to start accounting for, and as a result budgeting accordingly.

___________________________________

*At least until Tim gently reminds me other folks are paying for those resources you are burning.

Playing in the Cloud: Integrating Etherpad with Jitsi

One of the first things Tim showed me when we starting using Jitsi internally was the ability to integrate Etherpad Lite. I wanted to give it a try given I am working on demoing fully open source replacement for Zoom/Google Docs/Slack with Jitsi/Etherpad/Mattermost. I am now officially 2/3 of the way there 🙂

So, I already have both an Etherpad and Jitsi app running in Reclaim Cloud thanks to our handy-dandy one-click apps in the Cloud marketplace. After that Tim shared this guide in the Reclaim Hosting Community forums for integrating my Etherpad with Jitsi, and it worked a treat. When you click on the “Open shared document”…

… and voilà, now you have a blank Etherpad page that anyone on the call can edit directly from the Jitsi browser tab.

So, I added the KaraOERke instructions to the document which folks on the call can edit (though read-only is an option), it also has a link or an embed code.


You also have the option to import/export text and HTML files right from Jitsi. So, effectively full Etherpad functionality within Jitsi.

I really like the way this works, and through Jitsi you can also livestream to Youtube, I have not found other options yet, but given it is open source software I am sure they are not far off. I would love to be able to stream the Jitsi instance directly to ds106.tv.

The last piece of this open source remote teaching trifecta is Mattermost, I am going to dig in some more on that and see would integrating Etherpad and Jitsi into Mattermost looks like. But until then, you can always try this out for yourself using the 14-day free trial at Reclaim Cloud.

Turn Off the Cloud Lights!

One of the things I have been doing over the last couple of months is tracking how many resources (known in Reclaim Cloud as cloudlets) each application environment requires. This is important because the more cloudlets you use the more you pay, so trying to be as efficient as possible is quite important.

A cloudlet = 128 MiB + 400 MHz. Or, the equivalent of a ridiculously fast personal computer circa 1996 or 1997.

Crazy to think, but true. For each cloudlet you pay a dedicated amount of, for arguments sake, let’s say $3 per month. So, an app that requires 4 cloudlets will cost $12 per month if used constantly and the resource demands do not spike. Pretty easy maths, no? But what about a video conferencing applications like Jitsi that you only need at certain times?

I have been playing with this since July 1, and I have averaged about 10 hours on Jitsi Meet over the last two weeks. I have gotten in the habit of turning off Jitsi after every use, and turning it back on 10 minutes before my next meeting. Turns out the average cloudlet usage for an always on Jitsi instance is around 8 cloudlets per month, or $24. But when I turn it on and off regular it has only cost me cost under $1 so far this month, so literally a fraction of the cost.

And that should be easy for us to understand as we begin to think of applications on the web more and more like utilities. We turn off our lights when we leave the room because we waste less electricity and save money, I think for certain applications this approach means being more resource conscious.

I had a similar revelation this morning as I was tracking resource usage, my blog average 13 cloudlets per month, or $39. This means for most folks hosting your WordPress blog on Reclaim Cloud would not be more cost effective, probably true for most other PHP applications like Omeka, Grav, Scalar, etc. Shared hosting via cPanel will still be kind because it is far less expensive and a $30-$100 per year gets you pretty much all the applications you can run within limits. The Cloud makes you pay for your usage per applications, so you can see how quickly that would add up. Even a low-trafficked WordPress site in Reclaim Cloud would require 4-5 cloudlets, or $12-$15 per month, and that is just one site and it is not including the domain—what a deal we give you with shared hosting! 🙂

On the other hand, high trafficked sites that a require a virtual private server or a managed hosting instance might find the Cloud a lot cheaper given they’ll only pay for those resources used, rather than paying for enough CPU and memory to manage the “what if…?” scenario. In this regard the $300 a month you spend for the worst case scenario could be significantly less if most of the time that server is using just a fraction of allotted resources, and that is when the Cloud rules—when it can allow you to seamlessly scale as you need  but only pay for what you use—just like our water, heat, and electric bills. Stephen Fry’s 5 minutes video comparing cloud computing to utilities usage from 2013 is still one of my favorite takes on the changing nature of resources usage with the cloud.

So, one think I did this morning is go through my sites on Reclaim Cloud and look at which ones I could turn off to save some energy. Turns out the test instance of Ghost I am running takes up 7 cloudlets per month, or $21, so that was a fine candidate given a CMS site like Ghost always needs to me on. So, I sitesucked the ghost.murderinc.biz and copied the HTML archived files onto my cPanel account and re-pointed DNS. After that I stopped the environment. I can still keep the Ghost instance on my account, but like Jitsi, it can remain turned stopped (or turned off) and I won’t need to pay for anything but storage until I decide to actually use Ghost. In the event I don’t use it I can simply delete the app and keep my archived HTML version. 
So, to push the house of the future metaphor even further, I spent the morning turning off lights in the rooms of my digital house that I was not using, and my energy bill will thank me at the end of the month 🙂

Ow3ned or Owned?

A week or so ago Lori Emerson shared the above Tweet by Kyle McDonald who was sharing a slide from a talk by Julian Oliver. I have been using it as talking point for Reclaim Cloud since. One of the beautiful things about this Tweet is it highlights the fact there’s a whole new generation of elegant, powerful, and open source applications that you can run as an alternative to all those “free” sites that only cost your freedom 🙂 I think folks might start realizing free is not free and that to truly control your data you have to spend some time, money, and professional energy to do so. Not everyone will, but for those that do it’s a real alternative that helps organizations protect their members from re-living the whole “do no evil” with our data, lord tech conglomerate.

While thinking through how we are going to roll-out Reclaim Cloud, Tim and I started brainstorming clusters of applications that folks might use, such as Etherpad, Jitsi Meet, and Discourse for your courses; or ShinyApps, R-Studio, JupyterHub, Jekyl, and Voyant Tools for Digital Humanists; or for an organization/department NextCloud, Mattermost, Ghost, and MailTrain, etc. You get the idea, frame groups of applications for targeted uses that begin to frame an open-source ecosystem that we can create not only one-click installers for in Reclaim Cloud, as well as focused professional development and expertise to support those tools. Which is why this graphic was so useful, it does a one-to-one comparison and it helps us focus what we can and should offer folks as a real alternative to the less than ideal status quo.

Then yesterday I had a call with the awesome folks at Michigan State University to show them Reclaim Cloud, and Kristen Mapes mentioned that she was interested in an open source alternative to her current teaching suite of Google Docs, Slack, and Zoom, to which the graphic brilliant maps as Mattermost, Etherpad, and Jitsi Meet—a perfect fit and we already have one-click installer for all of them 🙂

Interesting that they use the terminology ownership given that has caused some concern in the past with “owning” your domain. But in this case ownership is premised around the idea of self-hosting your applications in order to have increased security and control over your personal, organizational, or professional presence. It makes damn good sense to me, but I am biased 🙂

Reclaim Cloud Q&A

Yesterday Tim, Lauren, and I sat down to talk more in-depth about Reclaim Cloud, as well as use the occasion to respond to questions folks have about this new platform that’s currently in open beta.

018: Cloud Q&A

If you want a look at what we discussed before committing, below are a list of topics covered and questions from the community.

Laying the Foundation / Ice Breakers

  • For anyone hearing about this for the first time, what is Reclaim Cloud?
  • Can you talk about Reclaim Cloud in terms of the ‘trailing-edge’ technology that Reclaim Hosting already has? How does it fit in with existing products? Or does it?
  • You all have no doubt heard about the ‘house’ metaphor being used to describe a personal domain or web space, in which each room of the house might be a page of your website. Or on a larger scale, each room might be a domain or application that makes up a greater digital identity. How does Reclaim Cloud fit into this metaphor?
  • Who is Reclaim Cloud for?
  • Talk to me about the technology. In checking out the reclaim.cloud website or even playing around in the Reclaim Cloud dashboard for the first time, it seems like there’s so much that can be done. And when getting into the world of Docker containers the options truly feel endless. This is great, but what are you all doing to guide the non-developers / beginners for getting started?

Questions from the Community:

Paul Bond: What’s in it for #ds106?

@Scottlo: What’s in it for me?

Ben Rimes: What would it take to launch and host your own video sharing platform; MyTube so to speak.

Shannon: You know I’m definitely interested in future thoughts about how this fits in with your work with schools doing DoOO. We have lots of CS majors who would benefit from this kind of offering I think.

Meredith Fierro: What’s one aspect Reclaim Cloud that has you all excited?

Anne-Marie Scott: Any plans for a Jupyter Notebooks installer? (on a related note, Tony Hirst – I made some notes on a simple 1-click installer he built for Reclaim Cloud)

As usual with discussions like this one, they’re as much about trying to understand what Reclaim Cloud could be while remaining cognizant that is will be an evolving narrative. more than that, we really hope to foster a community of users through Reclaim to start exploring the cloud in higher education through hands-on exploration and experimentation.

Adapting to the Reclaim Cloud

Tim has been on a full blown roll creating one-click apps for Reclaim Cloud. in just the first few weeks we already have more custom one-click apps in Reclaim Cloud than we created through Installatron for our first few years. It speaks to the power of this new platform for sure, not to mention the seemingly limitless possibilities it provides us to leverage Docker compose files for a whole host of applications will prove a huge boon for the marketplace of apps.

In fact, J.R. Dingwall’s recent post about trying to get the application Adapt Learning authoring tool installed on Reclaim Cloud speaks quite pointedly to the limits and possibilities of this new platform. Let me start with the limits, a lot of us edtechs are not sysadmins, so spinning up your own server, even if in the new fangled Cloud, is not necessarily simple:

Installing the authoring tool requires access to commandline, which I had never used before. Reclaim Cloud makes accesses command line super easy and clear, but like a foreign language, you gotta know what you’re doing. I also discovered that my approach to following instructions is not really the best way. Initially, you need to have four things in place to install adapt authoring: git, node.js, MongoDB, and grunt. That was a big hurdle until I found in the documentation how to check each. So I spun up the environment (selecting node.js, and mongdb) and then spun my wheels trying to figure out how to get git and grunt installed. Doh! Turned out they came with the package.

You can create just about any stack on Reclaim Cloud, but that process assumes certain skills like command line knowledge of Linux environments, how these next generation apps are packaged, not to mention their various relations, etc. I think this could be an environment edtechs become more and more familiar with, but I also understand the hard limits of entry, the need for support, and the time it takes for such specialized learning. SO, on the other side, the limits are real and it is up to us to try and make the tool accessible not only to the sysadmins, but also the folks who want to focus on its use. Here is JR’s second point really resonates with us:

I recall David Wiley’s keynote presentation at OER 18, a talk where he was asked to be provocative. One of the things he mentioned was about the days of compiling his own code, and while open source is very important that it is more important to make tools usable to the widest possible audience (I’m paraphrasing). I think Reclaim has often struck a great balance between providing simple easy to use access to tools and letting them get under the hood. Reclaim Cloud takes it to the next level.

I think this is absolutely spot on. Reclaim Cloud, as we are imagining it, gives the most Mountain Dew addled sysadmin an endless playground of possibility while at the same time providing a space for instructional technologists and designers to play and conceptualize the possibilities of this new environment, while at the same time providing focused community support and help when and where possible to make various technologies heretofore unimaginable just a click away.

Special thanks to JR for taking it to the blog, JR, I really appreciate the time and energy he spent illustrating the challenges and rewards of diving into a whole new paradigm for exploring open source edtech tools.

Reclaim Cloud’s Free Public Beta Now Open

I’ve been writing a bit about Reclaim Cloud on this blog over the past month in preparation for this: the launching of the open, public beta for Reclaim Hosting’s new cloud hosting service! It’s really exciting to share this news, and I would be lying if I didn’t admit I’m a bit surprised how quickly everything came together from the point we realized it was possible in late April to rolling it out publicly today.

What is it? Reclaim Cloud is a container-based hosting solution that allows folks to create custom technology stacks in everything from PHP to Ruby to node.js to Go —not to mention the possibility to load and run just about any Docker container freely available on the web. What’s more, we have a growing collection of one-click installers for a wide range of applications.

So, in short, it’s everything we have not been able to provide in terms of hosting a wide variety of technologies and tools beyond the LAMP stack. What’s more, the infrastructure allows the scaling of computing and storage resources seamlessly. A watershed moment for Reclaim and an important move to ensure we can help our community make sense of the ongoing shift in the hosting landscape.

How can you get help? Throughout the free, open beta period this month we’ll be handling all support through our community forums where you’ll find useful guides, focused video tutorials (like those featured above), and the ability to request help via forum topics.

Where do I sign-up? You can read more and sign-up for the free beta period on our website at http://reclaim.cloud

So, reclaim your sense of edtech exploration and jump on the next shuttle to Cloud City!

Mounting Directories and Copying Files between Containers in Reclaim Cloud

As I was moving both bavatuesdays and ds106 from a WordPress Cluster setup to a more simplified LEMP-based container, I was running into issues with private and public keys making it hard for me to rsync between containers. I posted my issue to a thread on the Cloud section of the Reclaim Hosting community forum (which will open up tomorrow, and will be where we will be handling support for the open beta period of Reclaim Cloud), and learned a very cool trick, namely that you can mount any one of your containers onto another environment and simply copy over any files you might need. For example, I need to copy the 15 GBs of files in bavatuesdays wp-content over to the new install, but with rsync not working the prospect of downloading and re-uploading seems time consuming and ridiculous.

“But,” to quote Tim in the forum thread

…there may be an easier way, you can just mount your older storage container to the new container like so:

Then you can simply copy files as if both folders are on the same server from /var/www/webroot/old-site to /var/www/webroot/ROOT

And it worked, what’s more we tried it with a database container after I took a dump of the ds106 database, and it was cool to discover you can mount a database container onto an add container to copy files as well.  So, transferring files and data between containers across different application environments is quote simple.

You go to the Config icon in the container you want to mount the other container to:

Create a directory in the /webroot folder, in this example old-site:

The click the gear icon and click on the Mount optin:

After that you will need to choose the container you want to mount, and include the path to the directory you want to have access to:

It happens quite quickly, the few minutes is an exaggeration…

And once it is mounted you can copy or sync files between the old-site folder and the directory you need to copy your files to (in this example /var/www/webroot/ROOT/) and it the copy seamlessly—without the wait.

Just another reason I am loving the Reclaim Cloud.

Holy Clustered Cloudlets, bava!

The last month needed to be a deep dive into the Reclaim Cloud, and I think I have held up my end. I migrated quite a few old sites, namely bavaghost, this blog (a couple of times), ds106 (a couple of times), and ds106.club. I also spun up some new applications, such as Jitsi, Discourse, Azuracast, and Minio. It has been a really rewarding, if at times frustrating, month. I am learning a ton about containers, Docker, and power of virtualized environments. I’ve been able to experiment with different database types and environments for bavatuesdays and ds106 simply by cloning an entire stack and testing the changes, this is illustrated nicely by my last post about switching database types on ds106 and down-sizing this blog to a non-clustered WordPress instance. I followed up on that last night with moving ds106 to a non-clustered WordPress instance as well, and like this blog it’s running cleanly on a fraction of the CPU resources.

The Reclaim Cloud measures resources in a unit known as a cloudlet, which is a measure of CPU usage or, 128 MB per cloudlet. So 8 cloudlets is 1 GB of CPU, 16 cloudlets 2 GB, etc. Part of my experimentation over the last month was to explore WordPress clusters given this environment would be ideal for heavily trafficked WordPress sites. And while ds106 and bavatuesdays have a bit of traffic, they are not “high traffic” sites so reducing the regular cloudlet usage by roughly 60% translates into significant ongoing savings monthly.

While I can reserve up to 32 cloudlets, or 4 GB of CPU for my WordPress site to scale up at any time, I will only be charged for how much I use, which is on average 10 cloudlets. With the clustered setup I was being charged minimum25 cloudlets given it was powering a load balancer, 2 NGINX apps, 3 MySQL databases, a separate storage container, etc.

With the non-clustered environment base resource usage (cloudlets) is cut significantly because there are far fewer containers.This containerized LEMP environment with the MySQL database, NGINX app, and storage all wrapped into one is far simpler than the cluster, and while it won’t scale as seamlessly as the architecture in the first image, chances are it won’t need to. And even if it does get a spike, the container can scale another 22 cloudlets, or almost 3 GBs of CPU resources, before running into any limits.

So, as June comes to a close I have moved this blog around a bunch over the last 6 months: from cPanel-based shared hosting to Digital Ocean to Kinsta back to Digital Ocean and now to Reclaim Cloud. Digital Ocean was costing about $25 per month for a 4 GB server and Spaces instance, which is quite reasonable. Kinsta would have been $80 a month for container-based WordPress hosting, which was a bit rich for my blood. Running bavatuesdays on the Reclaim Cloud will cost roughly the same as Digital Ocean for a server that can scale up to 4 GB (although in practice it is only using 1-2 GBs of resource at most). And while there is no possible way we can pretend to compete with Digital Ocean on server costs, if we are able to keep pricing within the same ballpark that would be amazing!