Ow3ned or Owned?

A week or so ago Lori Emerson shared the above Tweet by Kyle McDonald who was sharing a slide from a talk by Julian Oliver. I have been using it as talking point for Reclaim Cloud since. One of the beautiful things about this Tweet is it highlights the fact there’s a whole new generation of elegant, powerful, and open source applications that you can run as an alternative to all those “free” sites that only cost your freedom 🙂 I think folks might start realizing free is not free and that to truly control your data you have to spend some time, money, and professional energy to do so. Not everyone will, but for those that do it’s a real alternative that helps organizations protect their members from re-living the whole “do no evil” with our data, lord tech conglomerate.

While thinking through how we are going to roll-out Reclaim Cloud, Tim and I started brainstorming clusters of applications that folks might use, such as Etherpad, Jitsi Meet, and Discourse for your courses; or ShinyApps, R-Studio, JupyterHub, Jekyl, and Voyant Tools for Digital Humanists; or for an organization/department NextCloud, Mattermost, Ghost, and MailTrain, etc. You get the idea, frame groups of applications for targeted uses that begin to frame an open-source ecosystem that we can create not only one-click installers for in Reclaim Cloud, as well as focused professional development and expertise to support those tools. Which is why this graphic was so useful, it does a one-to-one comparison and it helps us focus what we can and should offer folks as a real alternative to the less than ideal status quo.

Then yesterday I had a call with the awesome folks at Michigan State University to show them Reclaim Cloud, and Kristen Mapes mentioned that she was interested in an open source alternative to her current teaching suite of Google Docs, Slack, and Zoom, to which the graphic brilliant maps as Mattermost, Etherpad, and Jitsi Meet—a perfect fit and we already have one-click installer for all of them 🙂

Interesting that they use the terminology ownership given that has caused some concern in the past with “owning” your domain. But in this case ownership is premised around the idea of self-hosting your applications in order to have increased security and control over your personal, organizational, or professional presence. It makes damn good sense to me, but I am biased 🙂

Reclaim Cloud Q&A

Yesterday Tim, Lauren, and I sat down to talk more in-depth about Reclaim Cloud, as well as use the occasion to respond to questions folks have about this new platform that’s currently in open beta.

018: Cloud Q&A

If you want a look at what we discussed before committing, below are a list of topics covered and questions from the community.

Laying the Foundation / Ice Breakers

  • For anyone hearing about this for the first time, what is Reclaim Cloud?
  • Can you talk about Reclaim Cloud in terms of the ‘trailing-edge’ technology that Reclaim Hosting already has? How does it fit in with existing products? Or does it?
  • You all have no doubt heard about the ‘house’ metaphor being used to describe a personal domain or web space, in which each room of the house might be a page of your website. Or on a larger scale, each room might be a domain or application that makes up a greater digital identity. How does Reclaim Cloud fit into this metaphor?
  • Who is Reclaim Cloud for?
  • Talk to me about the technology. In checking out the reclaim.cloud website or even playing around in the Reclaim Cloud dashboard for the first time, it seems like there’s so much that can be done. And when getting into the world of Docker containers the options truly feel endless. This is great, but what are you all doing to guide the non-developers / beginners for getting started?

Questions from the Community:

Paul Bond: What’s in it for #ds106?

@Scottlo: What’s in it for me?

Ben Rimes: What would it take to launch and host your own video sharing platform; MyTube so to speak.

Shannon: You know I’m definitely interested in future thoughts about how this fits in with your work with schools doing DoOO. We have lots of CS majors who would benefit from this kind of offering I think.

Meredith Fierro: What’s one aspect Reclaim Cloud that has you all excited?

Anne-Marie Scott: Any plans for a Jupyter Notebooks installer? (on a related note, Tony Hirst – I made some notes on a simple 1-click installer he built for Reclaim Cloud)

As usual with discussions like this one, they’re as much about trying to understand what Reclaim Cloud could be while remaining cognizant that is will be an evolving narrative. more than that, we really hope to foster a community of users through Reclaim to start exploring the cloud in higher education through hands-on exploration and experimentation.

Adapting to the Reclaim Cloud

Tim has been on a full blown roll creating one-click apps for Reclaim Cloud. in just the first few weeks we already have more custom one-click apps in Reclaim Cloud than we created through Installatron for our first few years. It speaks to the power of this new platform for sure, not to mention the seemingly limitless possibilities it provides us to leverage Docker compose files for a whole host of applications will prove a huge boon for the marketplace of apps.

In fact, J.R. Dingwall’s recent post about trying to get the application Adapt Learning authoring tool installed on Reclaim Cloud speaks quite pointedly to the limits and possibilities of this new platform. Let me start with the limits, a lot of us edtechs are not sysadmins, so spinning up your own server, even if in the new fangled Cloud, is not necessarily simple:

Installing the authoring tool requires access to commandline, which I had never used before. Reclaim Cloud makes accesses command line super easy and clear, but like a foreign language, you gotta know what you’re doing. I also discovered that my approach to following instructions is not really the best way. Initially, you need to have four things in place to install adapt authoring: git, node.js, MongoDB, and grunt. That was a big hurdle until I found in the documentation how to check each. So I spun up the environment (selecting node.js, and mongdb) and then spun my wheels trying to figure out how to get git and grunt installed. Doh! Turned out they came with the package.

You can create just about any stack on Reclaim Cloud, but that process assumes certain skills like command line knowledge of Linux environments, how these next generation apps are packaged, not to mention their various relations, etc. I think this could be an environment edtechs become more and more familiar with, but I also understand the hard limits of entry, the need for support, and the time it takes for such specialized learning. SO, on the other side, the limits are real and it is up to us to try and make the tool accessible not only to the sysadmins, but also the folks who want to focus on its use. Here is JR’s second point really resonates with us:

I recall David Wiley’s keynote presentation at OER 18, a talk where he was asked to be provocative. One of the things he mentioned was about the days of compiling his own code, and while open source is very important that it is more important to make tools usable to the widest possible audience (I’m paraphrasing). I think Reclaim has often struck a great balance between providing simple easy to use access to tools and letting them get under the hood. Reclaim Cloud takes it to the next level.

I think this is absolutely spot on. Reclaim Cloud, as we are imagining it, gives the most Mountain Dew addled sysadmin an endless playground of possibility while at the same time providing a space for instructional technologists and designers to play and conceptualize the possibilities of this new environment, while at the same time providing focused community support and help when and where possible to make various technologies heretofore unimaginable just a click away.

Special thanks to JR for taking it to the blog, JR, I really appreciate the time and energy he spent illustrating the challenges and rewards of diving into a whole new paradigm for exploring open source edtech tools.

Reclaim Cloud’s Free Public Beta Now Open

I’ve been writing a bit about Reclaim Cloud on this blog over the past month in preparation for this: the launching of the open, public beta for Reclaim Hosting’s new cloud hosting service! It’s really exciting to share this news, and I would be lying if I didn’t admit I’m a bit surprised how quickly everything came together from the point we realized it was possible in late April to rolling it out publicly today.

What is it? Reclaim Cloud is a container-based hosting solution that allows folks to create custom technology stacks in everything from PHP to Ruby to node.js to Go —not to mention the possibility to load and run just about any Docker container freely available on the web. What’s more, we have a growing collection of one-click installers for a wide range of applications.

So, in short, it’s everything we have not been able to provide in terms of hosting a wide variety of technologies and tools beyond the LAMP stack. What’s more, the infrastructure allows the scaling of computing and storage resources seamlessly. A watershed moment for Reclaim and an important move to ensure we can help our community make sense of the ongoing shift in the hosting landscape.

How can you get help? Throughout the free, open beta period this month we’ll be handling all support through our community forums where you’ll find useful guides, focused video tutorials (like those featured above), and the ability to request help via forum topics.

Where do I sign-up? You can read more and sign-up for the free beta period on our website at http://reclaim.cloud

So, reclaim your sense of edtech exploration and jump on the next shuttle to Cloud City!

Mounting Directories and Copying Files between Containers in Reclaim Cloud

As I was moving both bavatuesdays and ds106 from a WordPress Cluster setup to a more simplified LEMP-based container, I was running into issues with private and public keys making it hard for me to rsync between containers. I posted my issue to a thread on the Cloud section of the Reclaim Hosting community forum (which will open up tomorrow, and will be where we will be handling support for the open beta period of Reclaim Cloud), and learned a very cool trick, namely that you can mount any one of your containers onto another environment and simply copy over any files you might need. For example, I need to copy the 15 GBs of files in bavatuesdays wp-content over to the new install, but with rsync not working the prospect of downloading and re-uploading seems time consuming and ridiculous.

“But,” to quote Tim in the forum thread

…there may be an easier way, you can just mount your older storage container to the new container like so:

Then you can simply copy files as if both folders are on the same server from /var/www/webroot/old-site to /var/www/webroot/ROOT

And it worked, what’s more we tried it with a database container after I took a dump of the ds106 database, and it was cool to discover you can mount a database container onto an add container to copy files as well.  So, transferring files and data between containers across different application environments is quote simple.

You go to the Config icon in the container you want to mount the other container to:

Create a directory in the /webroot folder, in this example old-site:

The click the gear icon and click on the Mount optin:

After that you will need to choose the container you want to mount, and include the path to the directory you want to have access to:

It happens quite quickly, the few minutes is an exaggeration…

And once it is mounted you can copy or sync files between the old-site folder and the directory you need to copy your files to (in this example /var/www/webroot/ROOT/) and it the copy seamlessly—without the wait.

Just another reason I am loving the Reclaim Cloud.

Holy Clustered Cloudlets, bava!

The last month needed to be a deep dive into the Reclaim Cloud, and I think I have held up my end. I migrated quite a few old sites, namely bavaghost, this blog (a couple of times), ds106 (a couple of times), and ds106.club. I also spun up some new applications, such as Jitsi, Discourse, Azuracast, and Minio. It has been a really rewarding, if at times frustrating, month. I am learning a ton about containers, Docker, and power of virtualized environments. I’ve been able to experiment with different database types and environments for bavatuesdays and ds106 simply by cloning an entire stack and testing the changes, this is illustrated nicely by my last post about switching database types on ds106 and down-sizing this blog to a non-clustered WordPress instance. I followed up on that last night with moving ds106 to a non-clustered WordPress instance as well, and like this blog it’s running cleanly on a fraction of the CPU resources.

The Reclaim Cloud measures resources in a unit known as a cloudlet, which is a measure of CPU usage or, 128 MB per cloudlet. So 8 cloudlets is 1 GB of CPU, 16 cloudlets 2 GB, etc. Part of my experimentation over the last month was to explore WordPress clusters given this environment would be ideal for heavily trafficked WordPress sites. And while ds106 and bavatuesdays have a bit of traffic, they are not “high traffic” sites so reducing the regular cloudlet usage by roughly 60% translates into significant ongoing savings monthly.

While I can reserve up to 32 cloudlets, or 4 GB of CPU for my WordPress site to scale up at any time, I will only be charged for how much I use, which is on average 10 cloudlets. With the clustered setup I was being charged minimum25 cloudlets given it was powering a load balancer, 2 NGINX apps, 3 MySQL databases, a separate storage container, etc.

With the non-clustered environment base resource usage (cloudlets) is cut significantly because there are far fewer containers.This containerized LEMP environment with the MySQL database, NGINX app, and storage all wrapped into one is far simpler than the cluster, and while it won’t scale as seamlessly as the architecture in the first image, chances are it won’t need to. And even if it does get a spike, the container can scale another 22 cloudlets, or almost 3 GBs of CPU resources, before running into any limits.

So, as June comes to a close I have moved this blog around a bunch over the last 6 months: from cPanel-based shared hosting to Digital Ocean to Kinsta back to Digital Ocean and now to Reclaim Cloud. Digital Ocean was costing about $25 per month for a 4 GB server and Spaces instance, which is quite reasonable. Kinsta would have been $80 a month for container-based WordPress hosting, which was a bit rich for my blood. Running bavatuesdays on the Reclaim Cloud will cost roughly the same as Digital Ocean for a server that can scale up to 4 GB (although in practice it is only using 1-2 GBs of resource at most). And while there is no possible way we can pretend to compete with Digital Ocean on server costs, if we are able to keep pricing within the same ballpark that would be amazing!

Running Azuracast on Reclaim Cloud

Following-up on my last post about Reclaim Radio, here is how I got the web radio software Azuracast up and running. The process was made easy by the fact that Azuracast has instructions on how to self-install a Docker instance of their software. Even better, Reclaim Hosting now has Reclaim Cloud that just so happens to allow you to install Docker containers quite easily.

To begin with you will need to setup a Docker Engine on Reclaim Cloud by clicking on the downward-facing arrow next to the Docker tab:

After that select Docker Engine:

At the next prompt create the domain (reclaimradio.uk.reclaim.cloud), name the server (Reclaim Radio), and decide in what region you want the app to live (UK).

After the Docker Engine is created you can login to the web SSH tool and create the azuracast within the var directory:

mkdir -p /var/azuracast

After that, change directories:

cd /var/azuracast

From within the azurcast directory download their Docker Utility Script:

curl -fsSL https://raw.githubusercontent.com/AzuraCast/AzuraCast/master/docker.sh docker.sh

Set it as executable:

chmod a+x docker.sh

And then run the Docker installation process:
./docker.sh install

Soon after that the container with the Azuracast software will be up and running.

Keep in mind, you will want to make sure you define the domain as the custom domain you want the software to run in, in our example it is reclaimrad.io version reclaimradio.uk.reclaim.cloud. This is important when installing the Let’s Encrypt SSL certificate through the application.
./docker.sh letsencrypt-create reclaimrad.io

Finally, you will want to point an A record for the custom domain name to the container’s public IP address where ever you manage DNS, for this instance I was using the Zone Editor in cPanel for reclaimrad.io.

At this point you can login to Azuracast and start managing the application, here is a good guide to get you started on that journey.

Jitsi on Reclaim Cloud

Zoom’s privacy record has been spotty at best for a while now, but recent news pointing to their shutting down activist’s accounts at the behest of the Chinese government is yet another reason to think twice before using that video conferencing service. As Zoom has taken pole position in the edtech industry along side the learning management system as schools seem unable to imagine teaching asynchronously, the idea of an open source alternative to Zoom seems welcome. BigBlueButton is an existing option that folks at the OpenETC use and seem quite happy with. Another I’ve heard a lot about more recently is Jitsi Meet, and it just so happens that Reclaim Cloud has a one-click installer for Jitsi so I spun one up for myself.

Blurring the background for that 3-D motion effect

Not only is Jitsi encrypted end-to-end, but it is also as intuitive and seamless as Zoom. It allows screen sharing, in-app sharing of YouTube videos, chat, hand raising, and full screen or tile view.

There are also speaker stats for clocking who talked for how long, as well as bandwidth indicators for each participant in order to help identify where any connection issues are originating.

There are also integrations with other applications, such as for communal editing of documents in Etherpad or connecting your Google calendar:

Rooms you create on the fly can quickly be secured by the host with a password to prevent Zoom-bombing, and as host you can set these parameters much like in Zoom.

Over the past two weeks we have used Jitsi internally at Reclaim Hosting and it has been seamless. We’ve had no issues with groups of 7 or 8, and one-click install in Reclaim Cloud can support up to 75 users, but if more spaces are needed the instance can be vertically scaled.*

Also, it is worth noting I was able to map the instance on a custom domain, and I now have yet another tool within the complex of my Domain that I can use as needed. Pretty slick.

One thing that is not possible with Jitsi on Reclaim Cloud just yet is recording the sessions within the instance. That is something we are currently exploring, and once that is possible I will be hard pressed to see the advantages of Zoom over Jitsi in any regard.

__________________________________

*Jitsi scales resources up and down based on usage (think of scaling light using a light dimmer) which means you only pay for what you use. What’s more, you can also turn off the instance when it’s not in use to save even more on resource usage, which is true of any application on Reclaim Cloud. Even when idle applications like Jitsi use a certain amount of server resources (what are termed Cloudlets), so turning off the instance until next usage is like turning off the lights in a room you won’t be occupying for a while to save energy and money.

Discourse in the Reclaim Cloud

What’s old is new again. In 2015 I wrote about Reclaim Hosting experimenting with the next-generation forum software Discourse using a multi-user Docker setup. We use Discourse for Reclaim’s Community forums and I’ve grown to love the software.* What’s more, as that 2015 post notes, it epitomizes some of the challenges of running these next-generation apps on existing, affordable commodity web hosting: namely it runs Ruby using Nginx and requires transactional email. As Tim pointed out yesterday, I never take the easy road with this stuff. Discourse, even on the Reclaim Cloud, is not a simple, one-click install. But I know folks asked me about it, so I wanted to see what the process looks like, and then document it, which I have in the Reclaim Community Forums (which will not be available for another two weeks given we are still building this out, but I will paste in below). I am also working on a video tutorial for this install.

The following guide describes the process of setting up Discourse on Reclaim Cloud, but before I paste it below a note for my particular case that is not necessarily generalizable. When mapping the forum it seems the proxied A record for discourse.bavatuesdays.com through Cloudflare (where I manage DNS for bavatuesdays) was creating issues. I needed to turn that off for Domain mapping to work:

If you are not using Cloudflare this issue should not occur, but it frustrated me for quite a bit.

This guide will take you through installing the next-generation forum software Discourse on the Reclaim Cloud using the official Docker image they maintain.

To begin with you will need to setup a Docker Engine on Reclaim Cloud, by clicking on the downward-facing arrow next to the Docker tab:

After that select Docker Engine:

At the next prompt create the domain (bava-discourse.ca.reclaim.cloud), name the server (Bava Discourse), and decide in what region you want the app to live (blame Canada!).

At this point your Docker Engine server will be spun up, and once it is you can login via the web-based SSH window provided to install Discourse:
From the command line you can install Discourse (and the command line will follow) but before you do you need to ensure 1) you have a transactional email account working and 2) an A record pointed to the container’s public IP address if you plan on using a domain other than mydiscourse.us.reclaim.cloud. In this example we’ll be mapping Discourse to the URL discourse.bavatuesdays.com.

Transactional email services, like Mailgun and SparkMail, allows you to setup email sending and receiving for apps like Discourse. For this example we used Mailgun, and the crucial information you will need are the SMTP server address, SMTP port, SMTP username, and SMTP password. Along with the domain name, this is the information you will be prompted for when setting up Discourse, and if the email does not work (i.e. is not verified through your transactional email service) you will not be able to use the application.

This guide for setting up a new email domain using Mailgun could prove useful. But keep in mind there are more options.

Once that is done you can now begin installing Discourse. From the command line run the following commands:

git clone https://github.com/discourse/discourse_docker.git /var/discourse
cd /var/discourse

After that you can launch the Discourse setup tool:

./discourse-setup

At this point you will be prompted for domain, email, SMTP details, etc.

Hostname for your Discourse? [discourse.example.com]:
Email address for admin account(s)? [me@example.com,you@example.com]:
SMTP server address? [smtp.example.com]:
SMTP port? [587]:
SMTP user name? [user@example.com]:
SMTP password? [pa$word]:
Let's Encrypt account email? (ENTER to skip) [me@example.com]:

We recommend defaulting to port 587 and skipping Let’s Encrypt account email if you do not want to receive email about the built-in SSL certificate.

This will generate an app.yml configuration file on your behalf, and then starts the install which takes a few minutes. If you need to change these settings after installation, you can run ./discourse-setup again (it will re-use your previous values from the file) or edit /containers/app.yml manually with nano and then ./launcher rebuild app, otherwise your changes will not take effect.

Last thing is if you are using a domain other than that provided by Reclaim Cloud you will need to go to Settings–>Custom Domains and add the domain there. Additionally, the domain you are pointing needs to have an A record pointed at the Docker Engine container’s public IP address.

After that, go to the domain and Discourse should be installed and ready to setup.

__________________________

*We are now running Reclaim’s Discourse forum in the Reclaim Cloud quite seamlessly.

Archiving ds106 docs

Part of moving ds106 to a new server is making sure you don’t leave a trail of dead links in your wake. With great classes come great responsibility 🙂 I think I have the caching issues and some of the kinks worked out after the move, but one think I did want to make sure wasn’t lost in the move was the ds106 wiki, also known as ds106 docs. It was used through 2014, and while it wasn’t a huge part of the class design, for quite a while we used it to for  tech tutorials, syllabi, and other assorted resources. For example, I forgot about the detailed tutorial I created for an animated series of Dead Zone trading cards:

Or the equally detailed Creating Animated GIFs with MPEG Streamclip and GIMP tutorial.

I understand these resources are not all that useful anymore, but the internet preservationist in me wants them to live on. There are other resources such as various syllabi for classes over the years, such for Alan Levine‘s and Martha Burtis‘s Camp Magic MacGuffin syllabus from Summer 2012, or the syllabus for the ds106 Zone in the Summer of 2013. What I noticed going through my early syllabi for ds106 is they were all the same, they just started riffing on a different theme as the years went by, but the core remained. And while that seems logical, I really didn’t remember simply copying and pasting the basics and then building the theme and the prompts of the class on the blog and through the assignments. So, all this to say keeping the wiki was part of the deal of moving the site off shared hosting.

One thing you realize when moving sites is the value of using subdomains versus subdirectories, let me explain. The MediaWiki instance was installed at ds106.us/wiki rather than wiki.ds106.us. That might have had more to do with the WordPress Multisite being subdomains and my not knowing how to resolve redirects, but if the wiki was installed in a subdomain it would still be live right now (which is probably a bad idea regardless). But given I moved everything in ds106.us and the wildcard subdomains to the Reclaim Cloud, I would not be able to run MediaWiki within a subdirectory of ds106. Whereas subdomain can always be pointed elsewhere, subdirectories lock you into the server you are pointing the root domain to.

So, realizing this I need to a) get the wiki up and running temporarily so that I could then use Site Sucker to get a full HTML-based file backup of the site. This is great for archiving and also ensures that the wiki will not go down as application versions change, modules break, or spammers find a way in.* As you can se from the Site Sucker screenshot above, there are files both in ds106.us/docs and ds106.us/wiki because we used the article path function in MediaWiki to have all articles resolves as ds106.us/docs as opposed to ds106.us/wiki, which explains why the root ds106.us folder has both /docs and /wiki and both have part of the HTML archived files.

Another thing I did before archiving the MediaWiki instance (which I also have a full backup of) was update it from 1.19.xx to 1.33.xx. I had to replace the MonoBook theme, turn off the locked-out module, and adjust some other errors as a result of the update, but I was happy and relieved that it worked after a couple of hours and MediaWiki was now running a supported version on PHP 7.3 no less. Part of me still loves the promise and possibility of MediaWiki, but after wrangling with the documentation and the code it was a good reminder why it was never sustainable-the interface and editing was never made any easier and versioning issues made long-term maintenance onerous.

And with that, I think the future-proofing of the ds106 infrastructure and trying to ensure there remains some link integrity is in pretty good shape. I’ll do another pass this weekend, and then terminate the shared hosting instance, and commit to the cloud!

________________________________________________

*While I was at it I took a flat-file back-up of all of ds106.us and got a database and file dump (as well as a full cPanel backup file) that currently live in DropBox. So, this is a note-to-self that I do have a full snapshot of the site from June 2020 when I go searching for it in the future.