jrobb.org

In the past it was a prophecy, but in the future the past has ocurred!

Hosting Some More Services

I have had a vps for some time now, and I use it for hosting my pump.io node and website mainly. Then, I fired up a minecraft server that I play on occasionally, and it runs there also.

I’ve grown tired of depending on SMS, facebook, or gTalk for chat communications so I decided to finally get around to installing an XMPP server. I found this link and the installation was pretty simple. The only problem was when starting the service, it was trying to bind to port 7777 on my first IP (of the two on the VPS), and this was already being used by hastebin. Easy enough, once I found this out with a netstat -lnp -t tcp — I could not config ejabberd to use a different port in the config file (that I could find, anyway!) so I just changed the hastebin config to use a different port. That was it! Then the XMPP server was up and running, and I created accounts for my wife and me.

So that was great, I did that yesterday.

The previous weekend, I was wanting my own solution to host some media and obviously there is mediagoblin for that. After figuring out a few problems and then setting up ssl certs, my mediagoblin was up and running.
This service is NOT on my VPS— I had an olderish desktop that was not being used, so I set it up and this actually runs from my home.

I figured while I have this server in my home, I may as well use it for a few more things. Oh yeah, and I was tired of using Dropbox…

Enter Owncloud! This setup went very easily, and in no time I had everything up and running, and syncing across my laptop, android phone, and tablet. I had one mishap where I had turned the “sync” setting to “ON” under android account settings, and that killed my battery— but aside from that it has been great!

I think those are all of the services that I have running at the moment, but it definitely feels good to be able to depend entirely on myself and free software instead of large corporate entities (which are constantly harvesting whatever information that we provide).

Mongodb Problem With X86

Very recently Dr. Sekula had posted on pumpio that his mongodb pumpio backend needed to be upgraded because of a crash. Turns out, once db size becomes large enough, the x86 mongodb doesn’t support it and will just crash — and apparently this was a design choice.

Maybe a day or two after this happened to him, my own pump (which runs on my vps) crashed and could not recover. My vps was running Debian 7 x86.

I made some backups and figured I’d just rebuild it with a 64-bit version. The images that my host (ChicagoVPS) has for x64 debian 7 are badly mangled and is riddled with dependency problems. While I could’ve installed Debian 6 x64 and then upgraded, I didn’t really feel like it.

I should mention that by this point I had rebuilt the server probably 5 times trying to fix the dep problems.

I looked through the other build options: many CentOS which is ok, couple of gentoos, the debians, fedora, and then some ubuntus. I decided to go with Ubuntu x64, and the newest version available was 12.10.

After getting it all set up, I went to restore my backups. Which is when I discovered that I had backed-up the wrong directory, and my backup only consisted of a bunch of symlinks and no real data. bummer.

Luckily, I had an encrypted file with my pumpio passphrase in it, and I also had a mongodb backup from a couple of days prior. These two things are the secret sauce to make the pump work without just rebooting the whole thing with a new domain and such.

I removed apache, which is weird and tries to bind to all of my IPs (I have two), and I installed nginx instead. Got mongodb running, and then installed my ssl cert stuff. Then got the pump up and going after git fetching the latest.

All went well.

Then later that night I realized that 12.10 was pretty old, and after looking it up I saw that support runs out in 2-3 weeks. d’oh!

The next day I ran the ubuntu upgrade cli tool and everything upgraded without a hitch, nothing was broken. It’s been up and running ever since.

Oh, and I’ve corrected my backup script so I’m not just grabbing a bunch of symlinks now. :–)

I’m super tired, so I’m going to head off to bed before I doze off. It’s Friday!

PS.

Here is a pumpio thread for commentation and such.

RasPi Pump: The End

So I’d written before and even did an HPR episode with @nybill about the Raspberry Pi that I was using as a pump.io server. It was running well, aside from being a little bit slow—and I did have a few reservations about memory consumption, which I had planned to “fix” by splitting the database serving portion of it over to a beaglebone black.

Around the time I was planning to do this, however, I started having some network problems at home and now most of my ports are blocked so the pi could no longer work as a social media server. I’m still having these problems, and haven’t worked out a way around it yet, so I decided to go ahead and shut it down.

Since then, I’ve used my backup account, and then most recently migrated back into using my server that runs on my VPS. That’s about where I am now, and aside from life getting busy and not being on the computer much lately, that’s pretty much the extent of what’s going on.

Raspberry Pi Pump Server

My favorite and most-used social network right now is Pump.io, which is created by e14n. Or, mostly it is by Evan. Evan is the creator of famed StatusNet software which identi.ca used to run before its pump conversion, and sites like quitter.se still run.

My Raspberry Pi Pump

Like most of the people on the pump.io network, I started on one of the e14n servers from the tryit link on the main project page (where there are several to choose from). Because this software is designed from the ground up to be federated, there are many different servers which all talk to each other (much like when you email someone, it doesn’t matter who your email provider is). Federation is great because even if one of these servers go down, the rest are still up and communicating—compare that to when something like twitter goes down and no on can access the service.

After a while of mooching off of Evan’s servers, I decided to start my own pump on my VPS.
This turned out badly for me due to some hacking nonsense that wiped my server. So I set up again on my VPS after it was rebuilt, at a different subdomain (so as to not confuse the webfingers).
After this, there was a bug that caused my pump to not receive any notices…which isn’t very social. (note: that bug is now fixed)

The Pi

Around this time I figured I could try to set up my Raspberry Pi and see how well it worked out, a few other guys had done it at this point (like sazius and didn’t have too many problems.

I set my Pi up such that it has a 4GB SD card that it boots on, which contains the /boot and / partitions. I then have a 32GB USB drive (that I found under the TV where my kids had hidden it some time ago) which is mostly used as /home, and 2GB used as swap.

I ran the sudo raspi-config to setup things like:

  • memory split: I turned the GPU down to 16, since I’m not using the video output at all
  • overclocking: I did “modest”, it seems stable but clocks a little faster than the default
  • changed root pw, keymaps, all that stuff

I also did some normal things:

  • created a new user for me to login as, and added it to the sudoers file (but requires pw)
  • uploaded my ssh keys
  • installed fail2ban
  • disabled root and pw login in sshd
  • setup a fixed IP on my local network
  • forwarded ports 80 and 443 on my router to the Pi
  • forwarded port 22 on the router as well, so that I can ssh in (or use mosh)
  • decided on a subdomain (make sure of this before you start following people, as webfinger is particular)
  • created SSL certificates for the subdomain I was going to use. How-to
  • created a non-priveledged user with no password just for pump.io (I think “pumpio”)
  • I am planning to run my pump binding directly to ports 80 and 443—if you have multiple machines using these ports you may want to look into a proxy setup.

Node

Luckily, there already exists a nodejs binary package for the Raspberry Pi—otherwise it would take forever to compile.

wget http://nodejs.org/dist/v0.10.2/node-v0.10.2-linux-arm-pi.tar.gz

Obviously sub your desired version in there.

Get Pumpin

It is possible that you can install Pump.io from npm like this:

cd /srv/
mkdir pump.io && cd pump.io/
npm install pump.io

However, this version can lag behind a bit—the master branch is the place to be:

cd /srv/
git clone https://github.com/e14n/pump.io.git
cd pump.io

Config

Then it is time to setup the config file. Well, I guess you could do it later, but I’ll talk about it here. I’ll summarize some things, but most of this and more is laid out in the Pump.io README.

{
    "driver":  "redis",
    "params":  {"port":6379,"database":0},
    "secret": "something secret",
    "noweb":  false,
    "site":  "jrobb/pi",
    "owner":  "jrobb",
    "ownerURL":  "http://pi.jrobb.org/",
    "port":  443,
    "hostname":  "pi.jrobb.org",
    "address":  "0.0.0.0",
    "nologger":  false,
    "serverUser":  "pumpio",
    "uploaddir": "/path/to/pump.io/uploads",
    "debugClient": false,
    "bounce": true,
    "compress": true,
    "key": "/path/to/pump.io/ssl/ssl.key",
    "cert": "/path/to/srv/pump.io/ssl/ssl-unified.crt",
    "disableRegistration": true,
    "noCDN": true,
    "firehose": "ofirehose.com"
}

DB Driver

As you can see here, I’m saying I’ll use the redis driver.

  • I tried using leveldb, but this failed some self tests and caused problems
  • The disk driver saves everything to disk, and is probably awful speed-wise
  • With the memory driver, nothing is retained at shutdown
  • MongoDB isn’t currently available on the Pi
  • Redis is pretty easy to configure and use

I haven’t had problems yet using Redis, but the memory usage is crawling up so it may be an issue on the 512MB RAM Pi at some point in the future.

Secret

This is what the pump uses to create a hash or something for the web sessions stuff. I’m not exactly sure, but don’t change this. And if you migrate your pump, keep this phrase if you want to use the same webfinger address, and maybe things will work out.

Address

IP address to bind to. In my case of using a dynamic IP, 0.0.0.0 is the way to go.

Install

After the config is setup, we need to install. This involves the databank you setup in the config.

cd /srv/pump.io/node_modules/databank/
npm search databank
npm install databank-redis
cd /srv/pump.io/
npm install

There is also the -g option, which is a global install. I opted for the local install. At this point, we should be good to go and ready to start the pump. Or try to.

cd /srv/pump.io/bin/
sudo ./pump -c pump.io.json

I run with sudo, but you could run it as root as well. As long as you’ve created the non-priveledged user and set it up in the config, it should drop down to that after binding to the port(s). If you don’t see any errors here, and you get a webpage loading at your web address, we’re in business.

One thing to note that is probably obvious: initially you’ll need to have the config “disableRegistration” set to false so that you can create an account. If you change it, you’ll need to restart the pump (ctrl+C, and repeat the above).

After the config is all set up and everything is working, I use forever to keep my node scripts running.

sudo npm -g install forever

And then to start the pump I do this (I have it in a bash file called startpump.sh) :

sudo forever -a -e /srv/pump.io/log/pump-err.log start /srv/pump.io/bin/pump -c /srv/pump.io/pump.io.json

And, that is about all there is to it.
Find a good client and some people to follow, and start pumping with a note or an image or even a reply.

A few more things

Commentationing

I welcome comments/suggestions/criticisms/questions/whatever :

There is a pump.io thread here and a StatusNet thread here

My Irssi Setup

I’ve never been much of a chat person, even IRC. Typically I would just pop in when I’m having problems. Lately I’ve been getting on IRC a bit more, initially to chat with people about pump.io bugs and problems, and then just to chat because there are some cool people there.

Anyway, I was using XChat to begin with, because it was installed by default on my LMDE install, and it is easy to use. XChat is great, however in the past I used irssi quite a bit and remembered liking it, so I thought I would give it a go.

And, the default configuration kind of sucks. XChat really has me spoiled with features. Luckily someone has already done all of this custimization work, so I can copy what other people have done and just grab the bits that I like.

I’ll never remember all this, so I am making a blog post if I need to set it up again later.

Installation & Getting Started

I use LMDE, so the normal debian way works.

sudo apt-get install irssi screen

I may be interested in some of the other things from the link above at some point, but I haven’t installed them yet. For instance otr:

sudo apt-get install irssi-plugin-otr

To learn how to use irssi including basic usage, their site goes about explaining a lot of it somewhere around here. Some basics to get started:

/nick mynick
/server ADD -auto -network NetworkName irc.host.com 6667
/channel ADD -auto #channel NetworkName password

Pretty easy. I switch channel windows with CTRL + left / right arrow, or CTRL + window number. Save your setup:

/save

There’s a good start to a config file. (which is in ~/.irssi/).

Themes

So there really are a ton of themes, I haven’t even begun to look through them all. I just went with the standard one from the post I was reading, http://irssi.org/themefiles/xchat.theme. Hey, XChat is comforting. I saved that file into ~/.irssi/ and loaded it with:

/set theme xchat

Then to save the setup, I believe it is:

/layout save

Scripts

This is where things get interesting. Basically, all of the cool things that XChat does for you that we all know and love… irssi does not do. By default, that is. There are a ton of scripts available to do a myriad of things, but I’ll only barely scratch the surface here. You can find a lot of scripts on Irssi‚Äôs official script site: http://scripts.irssi.org/

You can load scripts if you load the script into your ~/.irssi/scripts/ dir:

/script load myscript.pl

or, if you put them in your ~/.irssi/scripts/autorun/ dir like me:

/run autorun/somescript.pl

Nicklist

Not having a nicklist along the side was driving me crazy. There’s a fix for that. There are a few ways to do it, actually, but I’m the most familiar with screen, and this setup is straightforward (and easy).

To start with, I saved the nicklist file from here: http://scripts.irssi.org/scripts/nicklist.pl into my ~/.irssi/scripts/autorun/ directory. Basically you start Irssi inside of a GNU Screen window, then enable the script. If you don’t run it in screen, I don’t even know what would happen, but I imagine that it wouldn’t work.

screen irssi
/nicklist screen

Here is a guide all about using irssi with screen, which is useful for information as well. I’ll pull some stuff from here later on: http://quadpoint.org/articles/irssi.

XChat Nick Color

This script works together with the xchat-theme (as you can find if you scroll up a bit) and does two very important things. First of all, it aligns all nicknames to the right in the chat area which creates a vertical line with the chat messages. This makes it a lot easier to read. And secondly, it will give all nicknames a unique color so it is easier to see who sends what.

Download: http://dave.waxman.org/irssi/xchatnickcolor.pl

Hilight Window

Without this, your name (or other words) don’t get highlighted so you never really know. Now, with this plugin, a new irssi window is made and all of these messages are placed in there. That’s ok, but we can also permanently set this window to display over the main chat area like this:

/window new split
/window name hilight
/window size 4
/hilight -word yournick
/help hilight

Save setup again:

/layout save
/save

Download: http://scripts.irssi.org/scripts/hilightwin.pl More info: http://quadpoint.org/articles/irssi#hilight_window

Advanced Window List (AWL)

This one is pretty great. By default, I never knew what the irssi statusbar was saying—with this script we can change it up and make it decent. Download adv_windowlist.pl Put the script in ~/.irssi/scripts/autorun and run it:

/run autorun/adv_windowlist.pl

Upon loading, AWL will create new statusbars on its own. AWL is an updated version of the older chanact.pl script. AWL has many, many new features developed by Nei. It would be worth your time to read the comments at the top of the script to get a feel for what all you can do with it (an entire article could be written on the features of this script and how to use them).

We don’t need the “Act” statusbar thing anymore. It says “(Act: 2)” when activity happens on window 2, which I just found out, but with awl and these changes we won’t need that.

/statusbar window remove act

There are a bunch of settings you can look into, but I am keeping it simple.

/set awl

The settings I am currently using:

/set awl_block -14
/set awl_display_key $Q%K|$N%n $H$C$S
/set awl_display_key_active $Q%K|$N%n $H%U$C%n$S
/set awl_display_nokey [$N]$H$C$S

If you like the setup, type /save to keep it. You can revert to the old act setup using /script unload adv_windowlist and /statusbar window add -after lag -priority 10 act.

Auto Away

Who doesn’t want this? It will mark you as away when Screen detects that you are not active in the session. Download: http://scripts.irssi.org/scripts/screen_away.pl

/set screen_away_active ON
/set screen_away_message
/set screen_away_nick

This one isn’t quite working right for me, though.

usercount

This shows a count of the number of users in the selected channel. Download: usercount.pl

/script load usercount.pl
/statusbar window add usercount
# You can also add -alignment left|right option

Conclusion

As of today ( July 20th, 2013 ), that is my setup for irssi. I’m liking it quite a bit again, though I don’t know if I will stick with it… XChat is just so easy. This is nice though, and it is in a terminal, so I could install it on my VPS or any server.

Later on if I try the OTR or other features I will update this post.

Screenshot

Below is a screenshot of the (currently unbusy) #pump.io.

  • GNU Screen + the nicklist script: Along the right you see the nicks in the channel
  • hilight window script: At the top is a mention (note it specifies it was in a separate channel)
  • awl script: Along the bottom above the entry box it shows the channel windows I have open. Current channel underlined. If another channel has activity, it changes color

Conversation

I posted this to pump.io, so the conversation could be here

My To-do List Today

  1. add my pump.io account as a destination in brdcst.it
  2. drink lots of coffee
  3. test if this blog post goes out to the remote destinations
  4. listen to TDTRS Ep 22
  5. lunch w/ the family over at my parents’ place (belatedly for my birthday)
  6. build my wife’s new computer desk (corner desk)
  7. move my wife’s current desk into another room and setup the kids’ computer on it
  8. setup the kids’ computer
  9. reclaim my/the kids’ raspberry pi :–)
  10. do stuff w/ the raspi
  11. maybe check the brakes on my car
  12. wing it

Busy Day With the Family

Today was a very busy day with the family: first thing this morning we left around 8:30am and went to a small zoo where we rode through on a horse drawn wagon. When riding through the driver would stop frequently so everyone could get to feed the animals and things.

After that, we had lunch and then decided to visit a transportation museum that we had passed on the way there.
The museum was really cool—it was a major train yard from the 20s or 30s up through the 60s or so when it was phased out after diesels replaced steam engines.
The place was huge and took a long time to go through, there were a lot of things to see, and even several engines and cars that we got to go in.

We also got to go on a short train ride across the property, as well as go for a ride on the turntable (“Best turntable ride in the county” the guy said! haha)
I like trains a lot, especialy steam engines, so it was neat to see the history of the place.
They had some plane engines and some replicas too, of the Wright brothers’ plane for example.
Several more larger things were in another (also huge) section that was under construction.

The car building was great too— it went from wagons up through the 80s or so which was also cool. I think I just like all kinds of engines and stuff like that.

We headed home and got back around 6pm. Nice and slightly sunburned—and all of the kids slept just about all the way back. They’re still exhausted, actually. I’m sure they’ll sleep all through the night (and I will too).
I must say, though, they were really good about all the walking around— we did a lot of walking, and that’s not usually a lot of fun with small kids.

Well, that’s all I’ve got for now.

VPS Hackeraging

About 3 weeks ago my VPS was hacked and the culprit decided for some reason to wipe data from the VPS nodes. Who knows why, but it happened. As a result, my server was wiped out and I’ve only just now been able to log back in and start setting some things back up.

I was depending on ChicagoVPS’ offsite backups, however that was hoping too much. It seems I’ve lost most things—my statusnet database, my pump database, any uploads associated with those. I’ll also have to re-setup ttrss and whatever else I will need running. The good news is, I’m finally able to log in and do these things. It’s taken a really long time to get to this point.

For now, though, I’m going to take a break and get to bed.

Minecrafting Ordeals

NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO. Not again.

Another ogre—it busted all up in my house and blasted all of my things. As a little backstory, this is probably the 6th or 7th time that this has happened on this server in the last 2 weeks. The Ogre is part of the Mo’ Creatures mod, which is otherwise pretty awesome. I am not a big fan of the ogres.

Oh, it isn’t that they are very tough to kill or that some of them shoot fire or anything like that.
Simply, for some reason they will spawn nearby my house, smash the walls, and destroy everything that I have accumulated in my chests.

This particular time wouldn’t have been so bad to start with, except that I was also being persued by an ultra rare werewolf and a rare spider.
(this means they have powers/magic and usually drop extra things).

So once the ogre busted down my walls, these two guys poured in and killed me with extreme predjudice. Over, and over, and over as I kept respawning. All while the ogre smashed my things in the chests…

Still, that also was not too terrible, except that I had just been on a killing spree at night— got like 5 ogres and 3 werewolfs (golden swords kill them in like 3 hits), and i had a ton of iron and things from my mine in those chests.

Also, not terrible I can deal with that….but then after I’d patched up my rubble hut with some dirt and sand I was just waiting for dawn to come (because my bed was also destroyed), and I was hanging out in the dirt walls.

Then out of nowhere some OTHER ultrarare werewolf who was throwing those exploding fireballs started shooting my mud hut.
Did I mention that I was behind a wall (of dirt)?

He was blasting through pretty quick so i ran up a ladder to my second level, but he kept blasting.
The wood and bamboo of my upper levels caught fire, and I only had a couple of sandstone blocks in my inventory which I used to block myself in as well as I could, but that didn’t work because i only had 8 blocks and the house was burning around me.

All the while he continued hurling those balls…from DOWNSTAIRS where he couldn’t even see me (still).

At this point, while the house was just burning down around me and all that was going on…I just rage quit. The first time this has ever happened with minecraft.

The ogre-ing was not happening again today.

…I will rebuild. With some other material that does not attract Ogres.

Google Reader Replacement

As was announced not too long ago, Google Reader is going away, but luckily there are many alternatives. There are even many free & open source alternatives for RSS readers—and I think this number will greatly increase now. In my opinion, the Goog closing down this project will spur innovation in an otherwise stale segment—and I think partially because of that, and partially because of the exodus, some people will discover some open source software that they love.

And that is always a good thing. … (Obligatory link to stallman singing the software freedom song.)

Anyway, I heard/saw many recommendations for Tiny Tiny RSS (tt-rss), and decided to give it a go on my newly setup VPS. Installation went pretty smoothly because I already had most of the dependencies installed for status.net. I’m using mariadb instead of mysql, but it’s a drop-in replacement.

Overall, I love it and so far I do not miss google reader at all. I’ve already stopped using it, and have fully been using tt-rss running on my own server (which has the added benefit of allowing me to own/control my data).

There are a couple of things that bug me, but it is kind of foolish to complain too much about an open source project—you can always make changes or create your own project.

  • It is written in PHP. PHP used to be amazing, but now it’s just a bloated language that is everywhere.
  • It requires mysql db! It seems this could be done without the db (Given I haven’t really thought about or looked into it).

Either way, many other solutions are out there and some are very different. Some are closed source.
The main thing is, we can still get our RSS feeds without the goog.

and even better with some FREEDOM.