Request more users

I received mail earlier today from Google announcing support for DKIM for all Google Apps editions. My boss has always complained that he couldn't send email to or receive email from his boss, and the IT people at his boss' company always refused to check their in-house email server for problems and keeps telling me to check my email server for problems.

Occam's razor moment. An in-house email server with hundreds of users vs. Google Apps. What are the chances the problem is on my end? DKIM doesn't really have anything to do with inability to receive mails, but I like to throw acronyms at my boss, so, I enabled DKIM on Google Apps and set my DNS accordingly, and on another tab, I was doing user transitions to the new Google Apps infrastructure. While I was doing this, I suddenly noticed on the Dashboard I can request for more users. The request for more users button was disabled a few years ago when Google Apps started the Premier Edition (now simply called Google Apps for Business) and since I already have 200 users I never actually needed more users. (Unlike new Google Apps users that can only get 50 users, I've been using Google Apps since 2006 and started with 100 users, and I've requested for 200 users in 2007.)

To make sure I wasn't imagining things, I googled and found this TechCrunch article, which is exactly what my dashboard looked like earlier. And just to make sure I'm not dreaming, I immedidately requested for 100 more users.

Yay!

Loving Linux Mint

(Continued from Part 2)

After trying openSUSE and Fedora and getting frustrated, I was almost ready to give up. Surely if people are trying to get Linux onto desktops, it should be easier than this? All those people that are saying Linux is ready for personal use must be delusional, or have really simple needs.

Anyway, my next was Ubuntu. Ubuntu is by far the most popular Linux distribution with a huge commercial backing. The installation screens of Ubuntu is also different from others in that it highlights Ubuntu's features instead of showing technical messages that most people probably won't understand.

Ubuntu has multi-lingual support built-in, but maybe because they don't have a lot of Asian developers, their selection of pre-installed input method are more than strange, to say the least, and the most commonly used Pinyin input, ibus-pinyin, isn't available by default, and none of those Chinese input selections in the screen below worked in a useful manner.

You might also think I'm easily discouraged. The thing is, I have no problems with using Windows or OS X in my daily work. I don't think Bill or Steve are the devil, and I use the right tool for the right job. I can easily learn to compile my own kernel or drivers, but I want to see how difficult it would be for an average person to get Linux going, without getting all frustrated.

Like Fedora, Ubuntu supports brightness and volume OSD out of the box, as well as Wi-Fi and Bluetooth toggling using Fn-F5. Trackpoing scrolling and sensitivity can be installed and configured the same way as Fedora.

The big surprise was for Active Protection System, which I couldn't get to work in Fedora. In Ubuntu, both tp_smapi (called tp-smapi-dkms) and hdaps are available in the repo. After installing them from the repo, APS worked right away without additional configuration.

I was almost going keep Ubuntu when I went to uninstall Evolution. I don't use a desktop mail client, so Evolution is no use for me, but when I tried to remove it, Software Center also wants to uninstall ubuntu-desktop as well, which is probably not a very good idea. Evolution is just too integrated with Ubuntu to make a full and clean uninstallation possible.

Next and final in queue, Linux Mint.

Ubuntu and Linux Mint are almost exactly the same thing. Linux Mint is based on Ubuntu but with somewhat different pre-installed applications and themes (Thunderbird instead of Evolution, Pidgin instead of Empathy, and it's green instead of brown), the partner repo is pre-enabled, and video codecs are pre-installed instead of auto-installation on first use.

Linux Mint's Software Manager is a lot like Ubuntu's Software Center. But it adds community reviews and stars, which sounds like a good idea at first, until you see the reviews for programs like Adobe Reader which is really useful for enterprise users like me, but typical "fanbois" would hate. I think this is Linux Mint's only drawback, and I wish there's a way to use the Software Manager without seeing those amateur reviews and scores.

In conclusion, I've been using Linux Mint for a couple of weeks now. As of this writing Ubuntu 10.10 is out, and I imagine a new version of Linux Mint would be coming soon.

(To be continued...)

Skype

7 million Skype users are online, and none of them are my friends. Skype has been down for almost 24 hours, and my phone has been ringing off the hook since this morning from desperate users who can't get in touch with their friends or customers. It's amazing for my users to realize how much they depend on technology when they're suddenly cut off from it.

I most definitely wouldn't want to be a member of Skype's IT team right now.

Technical PDF on the iPad

I've wanted an iPad since it was released to read my technical manuals in the PDF format (mostly because the Kindle didn't work so well). Unfortunately, I could find hundreds of iPad PDF viewer reviews online, but none of the reviews I found talk about the actual usability or display quality, and none of the iPad's I saw in real life have any PDF's loaded. I've walked up to strangers in public asking to see if they have technical PDF's in their iPad's, but all I've ever seen are games.

Since I couldn't find out the quality of iPad PDF displays and I didn't want to buy one blindly, so I gave up the idea of buying an iPad earlier this month. But of course, fate works in mysterious ways.

You need to click on the image to see the full size. The PDF's I loaded are Wrox's SQL Server 2008 Internals and Troubleshooting that I've been digging into lately, and a "borrowed" copy of Wiley's Mastering AutoCAD 2011 and AutoCAD LT 2011 so I can see display quality of images inside PDF. It has already been deleted as I'm not an AutoCAD user.

iBooks is the ebook app and store frontend from Apple, but it also handles PDF files. The display quality is good. But like most Apple programs, there's no way to customize the program at all.

Stanza is probably the most well known ebook reading app and it's available for every device. It handles PDF really fast, and there are all kinds of customization settings that helps the "reading experience". However, maybe because the program is optimized for real ebook formats such as ePub, the PDF display quality is really really poor.

GoodReader is perhaps the best known PDF reader, having separate iPhone and iPad versions. There are tons of customizations, but none of them seem to be really useful. I also hate that it has page flipping animation if you swipe to turn the page, but no animation if you tap to turn. However, the PDF display quality is by far the best. As can be seen in the images below, the graphics in the PDF can be seen clearly without zooming, and the AutoCAD image is smooth without jaggies.

ReaddleDocs is probably the most famous PDF reader since I see glowing reviews for it all the time and it's promoted everywhere. In fact, it was the first I tried. But it turned out to be really bad. Both the controls and image quality can't compare with any of its competitors. There is also a separate iPhone version available.

Fast PDF is not fast.

iRead PDF is the free version of iAnnotate PDF without the annotation features. iAnnotate PDF was popular for a while for being the first iPad PDF app that allows annotation simply by writing and marking with your fingers, but most of the competitors have now added similar features, and it just doesn't stand out any more. Also, it seems to be geared toward annotation and not reading, so the "reading experience" is lacking. There's no capture of the AutoCAD book since I just didn't care about the program enough to keep going.

pdf-notes is a free app that allows annotation. But it's slow and doesn't handle PDF display that well at all. Once again, there's no capture from the AutoCAD book.

It's quite obvious that the display quality of embedded graphics is greatly enhanced by anti-aliasing the graphics. But I looked and none of the apps have a setting to toggle anti-aliasing. It's either there (iBooks and GoodReader) or it's not (the rest), which is why iBooks and GoodReader are the best choices for PDF with embedded graphics.

Finally, I have some photos taken with my iPad next to the SQL Book. The first photo is the book opened to the same page as the screenshots above, and the second photo is opened to a random page that shows some code. The iPad is running GoodReader.

Some moiré is visible on the iPad screen due to the low quality screen protector film. This is not a screen protector film review, but that particular film is so bad that after two days, I already have permanent markings on the film on frequently used areas such as the area where I tap to go to the next page! And I don't know where they get the "anti-reflection" from. Maybe it reflects so much light you get blinded so you don't see any more reflections.

GoodReader is by far the best of the PDF readers I tried. However, I really don't like not having page animation when tapping to turn the page. The program is highly customizable, but the things that can be customized just aren't useful. But the display quality and speed more than make up for the shortcomings.

As for Stanza, I've been using it for a long time on my iPod touch to read ebooks and I've customized it exactly the way I like it. I would continue to use Stanza to read PDF files that aren't technical books. Unfortunately, since GoodReader can't be customized as much as Stanza, I've had to "un-customize" some of my preferred Stanza settings so the two programs can behave in the same way so I don't keep confusing myself.

iBooks is also recommended if you prefer to use Apple's ebookstore and don't want to customize anything.

Happy reading!

Windows Server 911

What could be worse than never performing backups? Server crashes in the middle of the backup process. Which was exactly what happened to me a few days ago, while I was backing up SQL data off our ERP server, and then the work day started and users started calling in the complain.

After the server rebooted, the Intel ICH RAID started rebuilding itself and was stuck at 33% and the ECC errors kept on climbing, which indicated a bad hard drive. The Intel console didn't say which drive was going bad though. Fortunately, the BIOS settings page did say that drive 0 was bad, so I replaced the drive with an identical one.

Unforunately, after the drive was replaced, it restarted the RAID building process, but the Intel Matrix Storage Console also complained that there's a missing drive. I kept wondering if I did something wrong, but luckily, after about 30 hours of RAID rebuilding, the process completed and everything was fine, and the missing drive error disappeared. (I didn't capture an image of the missing drive error, since I thought it would be there even after the rebuild was complete.)

So everything turned out to be fine. But during the 30 hours that the RAID was rebuilding, I took another full backup of the SQL data and restored it into my server's identical twin with the newer OS. So now I have two identical servers with different OS's and different versions of SQL Server.

The older server is Windows Server 2008 with SQL Server 2008, and the new, live, server is Windows Server 2008 R2 and SQL Server 2008 R2. After some testing, I decided to just take out the older server and replace it with the new one, since I've been wanting to upgrade to R2 for a while. So the drive crashing and stressful server rescue (and getting abused by the users) turned out to be a good thing after all.

Before:

After:

Oh, and what could be more stressful than bringing a new server with restored databases online? A database went missing. But this was a result of the bad information our ERP consultants gave me (i.e. I should always restore database backups using the ERP client and not using SQL Server), which further proves their uselessness.

The accidental DBA

A lot of DBA seem to be accidents. A search for accidental DBA on Google turns out millions of matches, and now I'm one of them.

A few months ago, when my Microsoft Dynamics NAV ERP project was about to go live, one of the last things we got stuck at was a process called adjust cost item entries ("ACIE"), which basically takes all the transactions you've made, and figures out the average or FIFO cost of a particular item in your inventory.

So I was looking at the adjust cost process, and it was taking a long time. A loooong time. A looooooooong time. A looooooooooooooooooong time. After a few hours, I killed the process and decided to ask the consultant about it.

The mor^H^H^Hconsultant said something like... your server is probably way below our recommended specs, adjust cost only takes two hours on her notebook, you need to buy a new server, or install a modern OS like Windows Server 2003 and SQL Server 2005. I told her that our server already runs Windows Server 2008 SP1 and SQL Server 2008. Both are 64-bit. The server has a Core 2 Quad Q6600, 8 GB RAM, and loads of disk space in a RAID5 configuration. She then mumbled something about Windows Server 2008 is too new and probably incompatible with our hardware, but then she suddenly jumped and said that RAID5 is baaaaad for SQL Server.

Hmmf. I do know that RAID5 is not a very good choice to run SQL Server on, and our server really wasn't all that powerful in 2010 (it was bought in late 2007), but then the server was specifically bought to run the ERP software, our database wasn't that big, and we only have 20 users. Even if it's sub-optimal, it shouldn't take forever to do something that needed to be done weekly or even daily.

After trying a whole bunch of settings on both the hardware and on Windows, and being nagged to no end by the consultants that my RAID5 setup is no good for running SQL Server and that I need to buy the Enterprise Edition of SQL Server instead of the Standard Edition, I finally brought my second server with identical specs online, but I installed the database files on separate drives. It's not RAID10 like they asked, but it has separate drives for the OS, for SQL Server itself, for the database files, for the log, and tempdb. I also installed Windows Server 2008 R2 and SQL Server 2008 Enterprise.

So I started the adjust cost batch process on this server, and exactly 24 hours later, it completed! I had never been able to complete the adjust cost process on the live server, since the database server can't accept posting entries while ACIE is running. But running for 24 hours wasn't exactly acceptable. (I could even accept 12 hours (!) since our working hours is 12 hours.)

I couldn't understand why the process only takes two hours on her notebook, so I decided to steal a look at the notebook when she wasn't looking to see how things were configured there. But then I discovered that she was running the native client version of the Dynamics NAV software. GAH!

Dynamics NAV comes in two different versions. The native version uses a file based database, and the SQL Server client version which, of course, uses SQL Server. They're very similar, but code has to be optimized differently depending on which version is used. I realized then that the idi^H^H^Hconsultants had absolutely no clue whatsoever about SQL Server, and decided to take matters into my own hands. (Okay, I had an idea earlier that they were clueless when they installed the entire database into my boot drive when the data drive had 1 TB of empty space, but they argued that I set up the drives wrong.)

Oh, I did get the ACIE issue resolved. While I was trying out different hardware settings, one of their other consultants (PURE GENIUS, by God!) mumbled something about bugs in the ACIE routine, and sent me an object, which immediately resolved the issue, and my ACIE time dropped from 24 hours to minutes. Proving that all the problems had nothing to do with my hardware whatsoever.

Since I was taking matters into my own hands (becoming the accidental DBA), I had to start learning first. I first went to Amazon and bought The NAV/SQL Performance Field Guide, which is targated at Dynamics NAV and Professional SQL Server 2008 Internals and Troubleshooting, which is targated at SQL Server in general. Unfortunately, since I was such a newbie at SQL, and I like Wrox books in general, so I also got Beginning Microsoft SQL Server 2008 Administration. (Since then, I've also bought Manning's SQL Server 2008 Administration in Action, which is by far my favorite right now.)

After burying myself in the books for a few weeks, I started working on the SQL Server. First, I discovered that the consultants had allocated 40 GB to the NAV database's PRIMARY file, and it was only using 60 MB. The data filegroup, however, was set to auto-grow and it was at 12 GB with 99% full. So I resized it into a more logical 128 MB PRIMARY and 16 GB data. They told me that I had allocated too much space when they were the ones that did the idiotic allocation.

Second, I discovered that the NAV log file was 200 GB. We've never done a backup using SQL Server, since the consultant insisted that we should backup only using the Dynamics NAV client and not through SQL Server. I backed up the log which cleared it out, and resized it to a more logical 8 GB. Since we only do daily backups and don't care about data loss during the day, I also decided to change the recovery mode to simple.

After the above, I made maintenance plans to make daily backups using SQL Server's backup function. Backing up using the NAV client by then took nearly an hour, and restoring the database took nearly three hours. Backing up and restoring using SQL Server only took about 5 minutes.

Next, The Dynamics NAV/SQL Performance Field Guide provided an index defragmentation script, which took more than an hour to complete on the first run. I guess it was an indication how fragmented my database was. Nowadays it takes about 2-3 minutes to run.

The consultants also said that my server has way too little RAM (it has 8 GB) since task manager indicates that SQL is using all the memory. 8 GB is not that much for SQL Server, but we have a very small database! Besides, SQL Server is designed to cache data in memory and it's normal for SQL Server to use large chunks of memory. But I still made changes to SQL Server so it doesn't automatically take up all the memory.

I wrote above that the ACIE issue was resolved. However, the ERP software was still suffering performance problems, and the assw^H^H^H^Hconsultants were still insisting that my RAID5 (and lack of RAM) was causing problems. By this time I've given up explaining to them patiently that my RAID5 had nothing to do with the sluggishness, since the ACIE problem was fixed by a simple object. (Anyway, just by pure chance, or maybe it was the Will of the Gods, one of the drives in the RAID5 array died and I ended up in a RAID1 configuration, which I describe in another post.)

After doing all that, the ERP software is still uselessly slow most of the time. What do I do in times like these? Why, throw more hardware at the problem, of course.

(Image source: Imagine-Net Technology)