Tuesday, 29 January 2013

It's a little wet in Bundaberg

Not getting much done the last week..
Not out of lack of motivation, it's just been too wet and depressing with the floods and stuff in Bundy.

My house is a good 5m above the peak water line, so we were pretty much safe.
And we were lucky enough to keep power and ADSL.

A large part of town was cut off, and a lot of people are dealing with crap right now.

Hopefully they managed to save some of the rum..

Here's some pics I took, it wouldn't let me upload anymore than these.

The bridge around the corner from my place

Floodwater can be rather grotty

Roads everywhere were cut off

Thursday, 24 January 2013

It's Backup Time!

This is a friendly reminder to Back. Your. Freaking. Data. Up.

Have you backed your files up today? 
Do you do it every day? 
Does your computer automatically do it for you? 
Are you _SURE_ your backups are working properly?

Because if you don't keep regular backups, your puny backup-less computers are all going to die horribly and all your precious unsaved data is going to hell. Well. computer hell anyway. 

They will die, not due to random chance, manufacturing faults, power surges, lightning strikes, Mean-Time-Before-Failure, solar flares or from being physically dropped, but because I said so! Yes I'm an evil b'stard willing hard drive crashes on all you backup-less philistines! :-P

(actually, your harddrives quite possibly might be dying from those other things too, in fact I'm pretty sure my ill-will towards your un-backed up data doesn't have that much sway in its demise. But I still find it funny when it happens, because I warn people all the time and they don't listen.)

If you're currently feeling all smug because you're already prepared, go check your backups are still working, and then go back to feeling smug.. Then you can snigger along with me over the fates of all those silly sausages who don't listen ;) 

The Consequences

Save yourself from shame, embarrassment and dispair in front of the IT geek at the repair store (or your ever-so-patient nerd friend), when he asks where your backups are. And stop getting mad at them for YOUR failure to be prepared, or them not being able to resurrect your data from the unpleasant depths of satans colon! 

How sad would you be if you lost all your precious photos of Junior and the antics of your cat? The home movies of Sally making her first steps, or of her barfing up on Granddad. 

What if you lost all your MP3's along with other piles of preciously hoarded crud scattered over your hard drive?  

Need I even mention how much it would suck to lose that assignment that's due in 3 days?

I'm sure you are all familiar with Murphy's Law (<--a good read by the way) and it's variations. With computer hard drives, you can pretty much absolutely freaking guarantee that all hell will break loose at the exact time when it is absolutely worst for you.

You'd be amazed as to how many extremely computer savvy people I've talked to that still forgot to ensure they had adequate backups, and were forced to do the walk of shame. And yes, that includes quite a few programmers and techies. (admittedly myself a couple of times, I was too dumb to learn the first time) 

It's one thing to know how reinstall your system and even recover files from a dying drive, but another to get back the hours wasted trying to restore that one damn super-important folder you forgot to keep a copy of. 

Time is precious. Disk space is cheap.

Backing up your data is not hard

Now please note that I'm not talking about business solutions, I'm talking home use stuff. But if your computers at work haven't even got the basics, you better take notes here and go have a serious talk to your IT support guys about it.

I'm not going to explicitly show you how to set anything up.. I'm just going to give my best recommendations here, and hopefully encourage some people to take action.

If your still unsure of what to do after this, have a talk to your techie friend/neighbourhood computer whiz/repair store guy for the full story. 

And also note I'm not talking about solutions for massive (but of course legally obtained) "media" archives. It's best to keep these larger files on separate hard drives dedicated to media so they don't clog up your normal backups. I have observed the most common and cost effective practice is to keep a spare copy of these media archives at a mates place... if you know what I mean. ;)

How much space for backups?

A rough rule of thumb is you need enough storage for 2-3 times the internal disk space of each computer you need to backup. So for example, a single computer with a 500Gb drive, a backup disk of at least 1Tb in capacity space would probably be enough for average users. Having more disk space available for backups allows for further backup history.

With bigger drives, you can use some of space on the backup drives for other things, but it's a good idea to partition the backup disk, so you don't go wondering where'd all the space go when the backup software fills it without telling you.

Backup Hardware

There are quite a few good solutions for backups storage, some even come with backup software or other nifty tools when you buy them, not that I've noted any of the included junk was any good.

External USB Storage
If you just have one computer, go get yourself a nice big external USB backup hard drive. (they are damn cheap) At the start of 2013 they came in sizes up to 4Tb.

Brands? My personal preference is Western Digital, but I know many geeks will spit vile hatred at me for suggesting they are acceptable. I've had my own plethora of bad experiences with other brands.

In reality, it's much of a muchness, buy whatever the heck you like and can afford, just don't expect it to last forever.

Windows XP may choke on drives bigger than 2TB. But that serves you right for running a 10 year old unsupported operating system. :-p Upgrading that old computer to Ubuntu or something costs nothing and allows you to run bigger drives.

I use a 3TB WD My Book Studio drive on my Mac, partitioned into 1Tb chunk for backups, and the rest as a data store for crap I don't really care if I lose. I'd have used a cheaper USB3.0 drive if Apple bothered to take standards seriously, but it definitely looks pretty next to my iMac.

Network Attached Storage
If you have multiple computers or just need something more clever,  get a NAS (network attached storage) server and plug it into your router. These are basically small computers that are designed to share data storage space on your home network.

They even have off the shelf external drives with ethernet ports or wireless now. Handy, and surprisingly cheap too. 

There are many brands to chose from, and none I can confidently say is better than another.. I suggest to a quick google for "NAS" and see which ones get the best reviews.

I've noticed some of the cheap single drive network drives don't have the capabilities to allow you to repartition the disk space. This may cause some problem if you want to use them with multiple computers, or want some extra space for other things too.

For the rest of my computers and devices at home I have a DLink DNS-320, equiped with 2x 2TB 3.5" SATA harddrives. It's a bit slow transfer-rate wise, and can't do as much with a decent sized external drive as it should be able to, but it works well enough for what we need here. 

Cloud Storage
You could always spend loads of dollars for extra space on a cloud storage account like DropBox/Google Drive/Amazon/etc. These services are like having a NAS box on the other side of the world, except someone else does the yucky maintenance for you.

Most encrypt your data, and provide a secure method of accessing it. You may need to take a manual approach to backups using cloud services, but I'm sure there is software that automates it. (Ubuntu's native backup solution Deja-Dup, integrates with Ubuntu One cloud service nicely)

I personally find cloud solutions better for "in case of fire" backups, like family photos and other precious documents and stuff. There are plenty of free ones that give enough space for most people for those kind of purposes. (By all means, keep the important stuff backed up at home too!)

Be aware if you keep files on a paid subscription cloud service, and you let your subscription run out, those files are gonna go bye-byes.

Drive lifetime?

Backup drives up and die just like regular harddrives. So it's a good practice to replace them every couple of years.. Your mileage may vary, some have to replace yearly, others get lucky and not so much. 

Old backup drives that are still in working order make good media backup drives.

This is probably the one advantage of Cloud Storage, because if the provider is doing things right, they factor all that into the cost and do the maintenance for you. I guess it comes down to whether you can trust them to take care of your precious backups.

Backup Software

A shiny new backup drive is not as cool as it could be if you don't have some decent backup software to go with it. It's best to have something that automatically deals with this for you.

You may find you only need your home directory backed up, but if you aren't the kind of person to keep all your goodies under one folder, or haven't the slightest clue where all your stuff is, it's best to look for whole system backup options.

Apple's Time Machine is so gooood and it's built in on all recent macs. 

Make sure to give it a dedicated partition on your new backup drive of at least 2 or 3 times your internal hard drives total space. More if you wanna keep longer history. 

You don't have to partition, you can give it the whole backup drive if you want to, just don't go wondering why it fills the whole 4TB or whatever up in a month or two, leaving no room left over for junk you don't want to store on your hard drive. 

(Partitioning is done in Finder->Applications->Utilities->Disk Utility)

Acronis True Image is good for all you (sad, loney, pretty UI deprived) windows users. From my days as a computer techie, this is one program that just WORKED as intended.

The latest two versions of windows may have enough backup capabilities built in, provided you take the time to learn to use them.

The majority of modern decent Linux distributions also have backup capabilities pretty much standard. If you don't find one already installed, there is a plethora of backup software in the package management tools. Search your distros software center for "Backup" and pick the one that works best for you.

Ubuntu's Deja-Dup looks pretty cool. (Pity the rest of the Ubuntu doesn't look so cool anymore. Unity sucks! :-p)

If your a geek, rsync your critical folders to your backup drive, but then again, I shouldn't even have to tell you this. In fact, turn in your geek card if you haven't got decent backups setup already.

And if your a software developer and not using a source revision control tool with an offsite server, I laugh at your hideously depressing loss of code. Muahahahahaha. 

In Summary

A properly configured backup solution requires little maintenance or thought. Just check it's actually backing up regularly, and don't forget to monitor your backup drives for errors along with your system drives.  

Do it right this year, and you too can snigger along with us other prepared people next year, at all those poor dumb schmucks who still don't get why backups are so important..

Monday, 21 January 2013

Texture caching OpenGL fonts

As I mentioned in an earlier post, reliable GL font rendering has been a major source of cross-platform headaches for me in SleepyHead.

What works perfectly on Linux will look like crud on a Mac, an will break on windows depending on the off-screen rendering method used.

Big mess, until I got unlazy and implemented a pixmap caching system to try and work around this, and the results were rather surprising.

The boring technical stuff
Previously, I was using either the QGLWidget::renderText() method, or using QPainter to draw text directly to the screen device. As I said, neither method looked good on Macs, and neither allowed for glitch free offscreen text rendering, no matter what Frame/Pixel Buffering method I was using.

In an earlier attempt to improve performance, all graph area text in SleepyHead is queued up for drawing until all the other graphics primatives are finished drawing, then the DrawTextQue function in my graph area gGraphView (QGLWidget derived) dumps this queue to the paint engine.

I'm still using this Text queue, however now, instead of rendering to the GL context/screen device, instead I create a QPixmap/QImage of the dimensions the text would be (using QFontMetrics to get the sizes). I then render directly to the pixmap using QPainter, which is then bound to a 2D texture, and drawn to the screen.

Now if I just left it there, all I've achieved is slowing things down dramatically, as it's creating an extra step for every element of text drawn. (Which does alleviate mac font problems, but still not acceptable)

Enter pixmap caching.

Qt provides it's own image caching solution called QPixmapCache, which would work fine if I wasn't using so much native GL code. It currently lacks the ability to unbind OpenGL textures when purging the cache.

So I wrote my own, which basically does the same thing, but unbinding textures on cleanup.

For each text element that is drawn, I construct a hash key containing the text itself, a text key describing the QFont, a string describing the color, and the antialias boolean setting.  Size and rotation don't matter here.

I search a QHash using this key, to see if there is a prior record and image available for this text item.
The relevant structure looks like this

struct myPixmapCache
    quint64 last_used;
    QImage image; // was originally a pixmap.. i'm too lazy to rename the struct
    GLuint textureID;

The cache is defined as following hash.
QHash<QString,myPixmapCache *> pixmap_cache;
If the hash key is not found, the text pixmap image is created, then bound to the records textureID, and a new item is saved in the QHash.

Now that the text element definitely is in the cache, so the 2D texture can be blitted to the screen device. I use QGLWidget::drawTexture() to do this.

Then the last_used field is updated with the current timestamp. This is so the cleanup algorithm, which runs at the beginning of this DrawTextQue function, if the pixmap caches memory requirements go beyond a set threshold.

I track all images memory usage roughly using the images width()*height()*(depth()/8);

To cleanup I scan through the pixmap_cache and look for outdated last_used timestamps (about 4 or 5 seconds old) and drop these keys to a QList, then afterwards go through the QList and unbind and delete the Pixmap cache entries, removing them from the Hash.

All this can be seen in qGraphView.cpp/.h

Performance Results
I implemented a switch so this new method can be switched on or off in the preferences, and used the "fps" counter that displays down the right side of the graph view, turned on in Help->Debug Mode.
(It's not super accurate counter, but it will do. it's a guesstimate of the FPS, not the real one, as SleepyHead doesn't have to update every frame unless it's got something to do)

The best improvement came in the Overview tab.. Scrolling over the AHI/usage/etc bar graphs.. The tooltip pops up and cruises smoothly at around 200-300fps. Perviously I was only getting 21fps and it the tooltip lagged behind miserably.  that is at least a 10 times improvement.

Vertical scrolling took a huge boost too, as all the Horizontal and vertical graph text gets redrawn from the cache each update. Much smoother. Went from 60fps to 300-400fps on my mac.

Vertical graph headers only ever change when graphs or the screen are resized, so it helped here too.

All in all, a big performance gain, for not a huge amount of effort.  

Pixmap caching can be used for anything that contains repeated graphics primitives such as lines, points and quads, however text was the biggest performance bottleneck I found in my application

Saturday, 19 January 2013

Olive branch to the ASAA

The following a copy of a letter I sent to the administrator of the ASAA forum apneasupport.org. In the spirit of openness I'm posting it here. It has seemed in the past that things I write to them get misinterpreted easily.

Thursday, 17 January 2013

Coding again.. SleepyHead needs an update.

Feels good to functional again, I'm writing actual code for the first time in a year.
Well, I should say it feels like I'm hitting my head against the wall, but from what I remember, that's fairly normal. :)

Struggling with a couple of annoying Qt bugs on the Mac platform at the moment.

There is a memory leak in QPixmap/QImage which causes crashes in rather silly places due to the way garbage collection is done. Of all the stupid places to crash it's in the Printer code. Consistently inside of the QPrinter::newPage() function, cleaning up graphics resources. Ugh.

To work around this one I think I need to order how pixmaps are dealt with in the printing pipeline so nothing gets garbage collected during printing.

Printing works fine on all other platforms of course.

Then theres the text rendering on mac.. It's been wonky for quite some time.
Have a look at this graph section below..

D day. I didn't sleep very well back then.

It might be considered a cool graphics effect in minor doses, but all text produced by QGLWidget::renderText or QPainter::drawText on a GL device looks like that on Mac.

When I render to a Pixmap, of course it looks beautiful.. Just cropped to the left is some nice straight text, because the yAxis tickers are rendered to a QPixmap when needed to update, and drawn from an image cached as a 2D texture.

I think I might need to try a similar approach with all GL text. Maybe a Pixmap caching system or something. Might even get a performance boost across the board if I do it right.

Also greatly annoying me at the moment is when I try to render text to an offscreen image, it simply doesn't show up no matter what method I use. How rude. This one only popped up on Mountain Lion, because QGLWidget::renderPixmap no longer works. I implemented a FrameBufferObject & PixelBuffer snapshot methods in an attempt to alleviate this, but still no good. At least I've got the Pie chart pack again.

And once again, works fine on Linux & Windows. In fact runs fine in a VirtualBox running Mint Linux on this very iMac. Faster, prettier and crash free too damnit.

Well, that's what's driving me insane codewise at the moment.

These problems are much easier to deal with now I have a Mac. Previously, I nearly worked poor Jimbo (James Marshall) to death testing all my mac code.. I don't know how he didn't go insane. I'm very grateful for his help. Somehow we cobbled together some working sleepyhead goodness on Mac.

Hopefully I can get enough of this sorted to get a new SleepyHead release out soon. A year overdue, but better late than never right?

Rich Freeman has sorted out all the PRS1 60 series patches and commited the code, along with some other bug fixes.. Man am I an idiot for not handing him the commit keys sooner. His work is very much appreciated.

Anyway. Back to work for me..

Wednesday, 9 January 2013

Oh yeah, the reason I've been quiet lately...

I haven't been very active during 2012.
I kinda got the wind knocked out of my sails.

This crummy, very short little animation I made in Blender 
might help explain the reason why.

On top of that I've had crud-loads of stress, depression, lots more headaches, a few bouts of atrial fibrillation, two mini-strokes which knocked the strength out of my left side for a while. Not fun.

Moral of the story, chose your life companions wisely, and even then, expect to be hurt some time in your life.

Being fiercely loyal does not mean even your closest friends will share the same trait.

If you have experienced this kind of callous cruelty from someone you love, what's left of my heart goes out to you..

If you've done this to someone in the past, I hope you truly learn to empathise and feel the pain you caused, so you never ever do it again.

If your doing this to someone who loves you right now, I hope you either own up, or go die in a fire.

There is no reason to be unfaithful in a relationship. If your genuinely unhappy, make your feelings known and leave, don't rip someone to pieces because you don't care for them anymore. They may feel a lot more for you than you think!

Apple Annoyances

Apple makes great computers. It's a just pity they are designed by art students and not programmers or engineers. A few things have driven me bonkers since converting, but luckily thanks to the many sages on the interwebs, I've managed to find solutions to most of them so far.

Except the first one, which I swear if apple doesn't fix it in a future version, so help me I will dig up Steve Jobs grave, reanimate him, slap him silly, and then send him back.

Ultimate Peeve #1: Mouse focus sucks 
Why the hell can't apple get this right by default?

Having been using blender a bit lately, this in particular is driving me insane:

When I've been working in another program on the other screen, or another window, then switch back to blender, I'm usually holding the alt key, because it's the "grab" modifier for the awesome but center-click button deprived mouse.

Mac decides it's best to hide the other programs window I'm working on, even though I'm clicking in another applications space. There is simply not enough swear words in the english language to express how idiotic I find that.

Steve Yegge covers this frustration nicely with an essay about this on his blog, and outlines a few of his attempts to rectify this.

There is also this post on trusty old StackOverflow, which highlights a few ways to alleviate the frustration.

Nothing yet has eradicated this evil. Please tell me if you know of some secret voodoo to annihilate this problem.

Peeve #2: Editing keys.
Being a programmer, I kinda use keys like page up/down, home, end, insert, and delete.. quite a lot when I'm editing text.

Apple computers of course are not designed by sensible programmer types, and thought how much shinier it would be without these.

The also put the function key to the left of the control key. Which would be damn near unforgivable if it wasn't for combinations of this key restoring a bit of normality.

Here's a short list of key combos I compiled for my sanity.

fn+Delete deletes forward 
fn+Up     page up
fn+Down   page down
fn+Left   goes to start of document
cmd+Up    goes to start of document (duplicate)
fn+Right  goes to end of document
cmd+Down  goes to end of document (duplicate)
alt+Left  move cursor back one word
alt+Right move cursor forward one word
cmd+Left  move the cursor to start of line
cmd+Right move the cursor to end of line

Thankfully the shift key to block select text works the same as on other platforms.
There does not appear to be insert mode. 

If anyone knows any other neat keyboard editing tricks, please let me know :)

Off topic, but I absolutely love these screen shot key sequences:

cmd+shift+4 then rubber band the area I want with a mouse, letting go of the mouse takes a shot
cmd+shift+4 followed by the space key, then click takes a shot of the active application window.

Both save the result to the desktop.

Peeve #3: The "dashboard" sucks tremendously!

Seriously.. No.. Apple doesn't let you have widgets on your desktop, you've got to have them on this stupid screen dedicated to them, and if you dare bump the top of the incredibly sensitive mouse the wrong way, it jumps there in it's hideous form with no remorse. EVIL!

Whoever invented that crap needs a slap upside the head.

Thankfully it's not hard to kill, as outlined on this awesome MacGasm page.

For those to lazy to click further, open up the Terminal app:

To kill it with fire, type the following:
defaults write com.apple.dashboard mcx-disabled -boolean YES;killall Dock

If for any bizarre reason you have regrets and want it back:
defaults write com.apple.dashboard mcx-disabled -boolean NO;killall Dock

Future Peeves
I'm sure I'll find plenty more annoying things as I go along, but since Mountain Lion, OSX is a lot nicer than the last time I used a mac, (which was probably Jaguar/Panther)

I certainly welcome tips on any of this kinda of stuff, as I'm still pretty much a noob on this platform.

2013 and I'm a Mac Convert.

Well, good riddance to 2012.

Turned out to be the worst year on record for me. Gah! Life Sucks! :(
Anyway, that's enough beyatching, but I definitely want 2013 to suck a lot less.

I've been a little bit too depressed/unfocused to write any code lately. SleepyHead has unfortunately been stagnating on my end. Luckily a few wonderful devs out there have been hacking away at it in my absence fixing various bugs and annoyances. I still need to get off my butt and help release a new build with the patches that are coming in.

Instead of coding, I've been spending my time trying to tune my artistic side a bit.

After years of Apple phobia and stirring up Apple fanboys, I got frustrated with my disorganized workflow, and bought myself a shiny iMac. Big mistake, because of course now I love it to pieces.

a little "music therapy"
I bought a copy of Logic Pro, so I can hook my musical instruments up and de-stress a bit.

Previously I've used Ubuntu Studio for this, and while it's definitely capable and very good, I just couldn't get it to run solid on any of my available hardware. I spent more time tweaking the sound engine then I did with the actual music.

Sometimes you just want something that works out of the box. I can certainly say Apple delivers on this one.

If your feeling game, I uploaded a few of my audio "experiments" to SoundCloud for anyone who wants to listen. Nothing fantastic yet. There's one short guitary bit I did with my old Ubuntu Studio system, the rest is just random Piano musings.. more will come later.

I've also started to dabble in a bit of visual art too, and being a computer nerd, I was drawn to the free and awesome Blender 3D modelling software. 

Took a few online crash courses/tutorials and am starting to get the hang of it. One in particular I found really helpful is this free Blender 3D design course by Neal Hirsig.

I'm finding it a very versatile and powerful program, I find it's a very "complete" package, covering everything from modelling to animations, physics and even has a game engine.

Here's a few renders from some of my recent "artwork" :-)

My interpretation of this neat cgcookie tutorial

Laura drew a rough sketch of this little guy
So I turned him into 3D

I did this from scratch with a few ladybug photos

The Mac is fantastic for this kind of creative stuff, but as with any new and different computer system comes a few annoyances, I'll post a few of my pet peeves with Mac along with the solutions I found next.