Like anything else, once you get good at programming a large part of your improvement comes in your ability to be more efficient.  Can you get the same results with less code, fewer files, less queries to execute, etc.  If you don’t, any inefficiencies really become magnified as a project scales and your hand gets forced.

When I whipped up Music-Alerts over a weekend last fall I really just wanted something that worked for ME.  I didn’t exactly plan it to scale.  I also wasn’t as good of a programmer as I am now.  When I made a database caching system for feeds, I figured I’d done everything I needed to for it to scale.

However, once I moved it over to our new server we started having issues.  Even after an upgrade to 4GB of RAM the script to generate the feeds was killing our MySQL server in terms of number of connections and number of queries.  While the caching system limited the number of times the web service connected to Amazon, it didn’t minimize the number of database connections.  For most sites this wouldn’t be a big deal, but RSS feeds get checked by feed readers several times per hour.  With 4,000+ feeds – and some subscribed to in multiple feed readers – you can do the math and see that our MySQL server was getting crushed.

So yesterday I spent a few hours building a static caching system that literally creates the feed as a file on the server and only connects to the database once every 3 days just to sync up with Amazon for new releases.  I also trimmed the core code from about 1,000 lines down to just over 100, eliminating a few unnecessary queries each time it does connect to the database. In less than 24 hours our bandwidth consumption, server load, and MySQL usage on the server have all plummeted.

Lesson learned: any RSS service I build in the future will be built with static feeds and minimal database connections.

In college, I downloaded music from our school’s network and used Winamp to play my music. I swore I’d never be one of those “fools” who actually paid for music and fell into the iTunes/iPod trap. Then I got an iPod for Christmas one year and realized I should start paying for my music (you know, since it’s legally and morally the right thing to do), so I fell in love with iTunes and to date haven’t looked back. Amazon’s affordable DRM free store is a great start, but they don’t have near the selection iTunes does. So I figured I’d always be an iTunes Store guy.

That is, until I started seriously playing around with Grooveshark. One of the best parts about Music-Alerts drawing some attention is that I’ve received praise from – and consequently struck up conversations with – some young startups in the music industry. One such new acquaintance is Andrew Wise of the upstart company Grooveshark, which was started by three University of Florida students. It’s so unique and has so many features, that I’ll just cut to the chase and list off what it does:

• There are two components – your online profile and the file-sharing software (like old Napster or Limewire) that accesses your MP3 files.
• You can listen to streaming music for free on the site as much as you want.
• You can create playlists, add friends, receive suggestions, etc (all the social networking stuff).
• When you want to download a track, you add funds to your account and buy it DRM free for $0.99. Royalties are paid to the labels, to Grooveshark, and to the person who you’re downloading the song from. Hence the slogan “everybody gets paid”.

After being invited to be a BETA tester, I just went in minutes ago and set up my profile and added $5 to my account. I then proceeded to search for a song, download it, and import it into iTunes (gotta be able to transfer it to the iPod). It worked awesome. Bottom line – as long as Grooveshark is able to strike up deals with all the labels and be legal, it’s now where I’m starting my music search. It’s a cross between a social network, p2p file sharing system, and streaming music service. For the same price as a song on iTunes, I get it DRM free and I get the advantages of a social network that knows my music habits (incidentally, if I have a feature request it would be to import my iTunes library XML file so it already knows what music I like).

The only real question is about the legalities. What are the origins of the music I’m downloading, and if 90+% of the music on the site is bootleg, how will record companies feel about it? On one hand, they might like the fact that they’re actually making money from it when they otherwise wouldn’t be. On the other hand, they might not like the fact that other people are getting a cut for uploading it. Only time will tell. I’m certainly rooting for it to work.

P.S. – the Music-Alerts Facebook App is REAL buggy. I know about it, I just hate debugging under the constraints of Facebook. I’ll fix it…eventually. My bad

With that, we needed a way to get out of the “techsphere” and into the mainstream…where most people don’t know what RSS is. The simplicity of Music Alerts coupled with the social nature of music preferences, makes it the perfect candidate for a Facebook Application. I haven’t really drank the Kool-Aid on Facebook apps like everyone else has, but I do see them as a valuable marketing tool for the right type of site.

The Music-Alerts Facebook Application actually expands a bit on the Music-Alerts site. You can install the application by going to http://apps.facebook.com/music-alerts/. Once installed, the app looks just like the Music-Alerts site:

After entering your list, your profile is updated with some of your favorite artists and their recent releases.

When you go back to the application, you now see all of your upcoming album releases and have the option to add/delete artists.

Of course, all of this means that I finally had to actually sign up for a Facebook account:

The app has a few bugs that I know about, but in most instances it seems to work fine. It’s hard to tell if the known issues are caused by Facebook (there are a lot of reports that the Facebook platform isn’t always stable) or by my code…so I’m going to wait to see if/when I should address them. Developing on the Facebook platform wasn’t really what I expected, but I am overall impressed with how they’ve set it up, and I’d definitely recommend that any PHP programmer spend a weekend learning it – the Facebook community is too powerful not to know how to develop for it. I only spent about 10 hours programming the MA app, and a simpler app could be developed in an hour or two. Tip: if you don’t have PHP 5 installed on your server, you’ll likely run into issues using Facebook’s PHP 4 Client Library like I did. This third party Client Library did the trick for me.

If you get a chance to take the app for a spin, I’d appreciate a quick comment below on your experiences.

Instead, I’m posting about Music-Alerts. After my little marketing challenge was done I expected traffic to taper off and traffic to level off. Then I could get back to “real work”. Instead – in true roller-coaster-week fashion – something inexplicable has happened: the site has worked it’s way UP the blogsphere and landed itself on giant tech sites like Lifehacker and MakeUseOf. Previously with iPrioritize I worked my ass off for months to get a mention on a big blog, and then watched the press mentions and traffic trickle down to smaller sites and eventually taper off. Instead, it seems like every day a bigger blog or site picks it up, and traffic just keeps growing. The site also now ranks top 5 for popular terms like”album release dates” in Google, which kind of solidifies the stream of traffic and means there won’t be a total crash once this PR wave stops.

The only explanation I can give is that it’s a really, really simple service that fills a need that was somehow unmet…and people love it. Honestly, it’s kind of cool and also kind of depressing at the same time. I mean, today Music Alerts will more than double the traffic for all of the rest of our sites combined! That’s SportsLizard + Detailed Image + Detail University + this blog + iPrioritize + the rest…all of which do pretty well for themselves. If traffic continues at todays levels, it will be well over 1 million unique visitors this month…wtf!!! Has everything else we done been so shitty compared to this? We’ve spent years on other sites and I spent a few hours on this one, and yet this one is really that much better???? Really makes you think.

My inbox has been flooded with “thank you” notes about the service, one person even going as far as saying “it’s guys like you that make the internet a better place”, when in reality all I was trying to do was make a feed for myself so I didn’t miss another album release. I even had a large Web2.0 music company contact us about integrating it with their application (we had a quick phone chat). It’s just been an insane day. This is probably the first project I’ve done where I really didn’t even think about as a business – aside from this blog – and it’s the most successful day (in terms of traffic) that one of my sites has ever had.

I don’t even want to think about what would happen if it lands on TechCrunch and continues to grow. At what point do you try to monetize something that is very tough to monetize…and is there a way to do it that doesn’t turn people off from the simplicity that attracts people to the site in the first place? Obviously, more affiliate links to buy/download + feed ads wouldn’t be overwhelming, but also probably wouldn’t make much money. Maybe the user base – which has grown by almost 1k today – will be worth something in and of itself…kind of like how Kiko made a business of growing a large user base and selling off.

Oh – and I now have an addiction to checking my stats every 10 minutes. I’m like a crack addict who can’t get off the high, but who knows he’s going to crash and crash bad. There’s no way it can keep growing…can it?

If those numbers keep up, this site will have a sizable user base in less than a year, and it may just have a chance at making us some $.  I think the next step will be to add more to each feed – include affiliate links to download the album and buy from several places other than Amazon, and also insert AdSense-like ads into the feeds.  I’d say that I’ll probably do that sometime in the next few months, and I’ll re-evaluate things if it really grows.  I love the simplicity of the site, so I’m going to try to work around making people register when monetizing it, which will likely limit what can be done.  Then again, keeping it simple and forcing those constraints might be what makes the site standout from all of the cluttered crap out there.

I did add one feature to the site today:  you can now add artists to your feed.  All you need to do is click the “Add to Your Feed” in the upper-right, enter your feed key (the last 8 digits in the feed URL), and enter the artist name.  A nice simple feature that addresses one of the main weaknesses of the site.

As I said back when I launched it, Music-Alerts is a site I’m really proud of.  I think it’s a service that a lot of people could benefit from, and I think I did a pretty good job on the design and programming.  That said, let’s be honest:  the chances for it to actually make us any money is very slim.  The only chance it has is to become HUGE so we could make money on advertising or affiliate sales.  Currently, the links to the albums are affiliate links, so in theory if a crapload of people used the service we’d make some money when they clicked through and bought an album.  But we’re not experts in the music industry and we don’t want to commit ourselves to a site with little to no revenue potential.

So while I want the site to be useful and become popular, I really don’t want to spend much time on it.  We’ve been getting some traffic because of the high search engine rankings, and there’s a good chance if I left it there it would naturally grow slowly but surely.  However, I still wanted to spend a little time and effort to see if I could get the site to catch the viral bug: you never know who will see it and have interest in it.

Today (Sunday) I finished my emails and daily tasks in an hour or two, and I had the rest of my football-watching day ahead of me.  I decided to spend exactly 1 hour marketing Music-Alerts…primarily because I wanted time to relax and watch sports, but also because I thought it would be fun to limit myself to such a ridiculously small amount of time (this way, if the site does take off I’ll look like a genius for only spending an hour marketing it ).  I also kind of wanted to prove a point that when your back is against the wall – either by choice or by circumstance – you can usually pull something impressive together.  While I really don’t expect much from my hour, I do expect an increase in traffic and feed creations.

Here’s what I did:

• Spent $100 on StumbleUpon PPC, which equates to 2,000 visitors to the site (5 cents/visitor).   I configured it to show to only people in the US that are between 18 and 30 and have listed “music” as one of their interests.
• Submitted the site to the QUIT (Quick Indexing Tool) on Blue Hat SEO.  The site advocates both white hat and black hat SEO techniques, so use your better judgment before following any advice on the site. However, it is a great read and I’m willing to take some risks with Music-Alerts so it was worth a try…I’ve been curious.
• Emailed about 10 influential technology blogs (like TechCrunch and LifeHacker) and a handful of popular music sites.  All it takes is one site like TechCrunch to cover the site and it’s ON.  Are the odds slim? Hell yea.  But it’s worth a shot, especially for the potential payoff.

I’ll follow-up sometime soon with the results!

Today I got my first real alert. It worked perfectly – Google Reader saw that the feed had been updated and notified me of the new item. I looked, saw the release of a new album (albeit an international release of an album already out), and I was like “wow, this is pretty cool”.

I also noticed that Music Alerts comes up #3 on a Google search of “music alerts”. Pretty cool to be just behind MTV and MSN after just a few weeks. I also find it satisfying that I’m now ahead of that stupid service Barnes and Noble offers where you can sign up for alerts for a whopping 30 artists.

Anyway, I’ll hopefully get some time towards the holidays to promote it. As long as it keeps working well for me, I’m going to be supremely confident in it and be as aggressive as I can with pushing it. We’re obviously not going to spend a lot of money on a side project (i.e. none), so I think the only real chance it has is to catch the “viral” bug and get caught up on the tech sites and blogs – certainly a possibility, but obviously not an easy accomplishment. We’ll see.

1. Faster feed generation.  Previously, I was generating the feeds on the fly every request…meaning each time you or your feed reader requested the feed, the system called Amazon and requested data back.  Aside from being inefficient, this took A LONG time and timed out the larger feeds.  My temporary solution was to build a “caching” system that stores the feed on Music-Alerts.com and checks for updates every 3 days.  The first time you load a large feed it might take a while, but after that you should be in the clear.  I’ll look into a better permanent system down the road.
2. Duplicate elimination.  I noticed my feed had a few duplicates in it.  I also noticed larger feeds had like 5 per album!  This was 50% my fault and 50% Amazon’s:  I had messed up on one of my loops and turns out if an artist didn’t have an album being released soon it showed the last one that did instead of skipping to the next artist.  On top of that, Amazon returns duplicates by nature as it is (do a quick search and see for yourself).  For that, I built a filter that compares album titles and weeds out any duplicates.

If you happened to build a feed prior to this post, it *should* work but your feed reader (which has the old one cached) might give you hell so it might be easier to just set up a new one.  The <description> tag in the feed has a comma-separated list of your artists so you can just copy and paste to make a new one…just remember to delete the word “and” before the last artist.

Whew.  This “weekend project” turned into a little more than that.  I’ll probably spend some time this Saturday or Sunday trying to market it a bit.  It’s definitely the type of thing that can catch on virally, so my hopes are I can get a TechCrunch or similar style tech blog to pick up on it…we’ll see.  I’ve got a few other tricks up my sleeve too…I’ll keep you posted if anything works.

Now back to “real” work.

I wrote on a sticky note “find music alerts site”. I figured later that night after I finished working I’d search around and find a good music alerts site to send me email/RSS alerts whenever my favorite groups released a new album. But when the time came, I couldn’t find what I was looking for. I sent Mike a message on Skype saying that it couldn’t be possible, there had to be something out there. Aside from Barnes and Noble (which sends email alerts for about 30 artists), I found nothing viable.

I sent him another Skype message saying that would be a great idea for a site. Ten minutes later I bought Music-Alerts.com. And about 8 hours of programming later I completed Music-Alerts.com.

You enter a comma-separated list of your favorite artists and the site generates a RSS feed of album releases for you. Using Amazon for the data, we add an “alert” to the feed as soon as the product is entered in Amazon’s database. The feed also shows any releases in the past 180 days in case you missed anything.  Each “alert” in the feed lists the band, album title, release date, and also shows the album art (if available) and links to the CD on Amazon.

It might shock you to hear this, but I’m more proud of this project than I am of anything else I’ve ever done. It epitomizes what it means to be a web entrepreneur in my mind. I had a problem, nothing available solved the problem, so I solved it myself. I’m banking that other people have the same problem, and if they do this site could become quasi-popular. On top of that, I think I did my best design/CSS work, and it’s by far the most efficient site I’ve ever programmed (the entire site consists of about 100 lines of code).

What do you think? I’ve obviously done limited testing, so please email or comment if you find any bugs.

P.S. – In a horribly ironic twist of fate, my personal music feed works fine in browsers, validates fine, works in every feed reader I try, EXCEPT Google Reader which is of course the one I use.  Other Music-Alerts feeds work fine in Google Reader, but mine magically doesn’t show up.  Aaaaah!

P.S.S – On about my 8th try it worked.  I didn’t change anything…just re-created the feed over and over again.  Stupid Google.