24
May 2013
AUTHORMike Benner
CATEGORY

General

COMMENTSNo Comments

Great Quote

I was reading this weeks Status Code Newsletter and the quote at the end was the following. I really liked it so I figure I would share it here. (Peter I trust you so no fact checking was done on this post).

“Computer programming is tremendous fun. Like music, it is a skill that derives from an unknown blend of innate talent and constant practice. Like drawing, it can be shaped to a variety of ends – commercial, artistic, and pure entertainment. Programmers have a well-deserved reputation for working long hours, but are rarely credited with being driven by creative fevers. Programmers talk about software development on weekends, vacations, and over meals not because they lack imagination, but because their imagination reveals worlds that others cannot see.”
Larry O’Brien and Bruce Eckel

24
Apr 2012
AUTHORMike Benner
CATEGORY

Development, General

COMMENTSNo Comments

Sometimes You Have To Go Backwards

Your metrics are showing a decrease in performance and users are starting to complain. You start digging through the performance metrics, source code and your stack to find inefficiencies and places you can gain that performance back. What if you do not find any low hanging fruit and solutions are going to be invasive, complicated and time consuming? While those things may need to be done, do not just dive in and start the surgery, stop and ask who am I doing this for?

Humans are driven by numbers. Why do you think there are some many top 10 lists, 5 easy steps to be a guru, etc? Most developers just start attacking the problem with the goal being to get those numbers back in an acceptable range. Maybe they through up some caching, rewrite some queries or upgrade a server. Ultimately they get the numbers back into that arbitrary acceptable range, wrote some fancy code and might have wasted their time.

Yes, I said wasted their time. We need to step back and look at more than the performance numbers. Look at the analytics for your app. What does the actual user usage look like? How does that data looked when overlaid on your performance metrics? Look for the patterns and correlation of the datasets. The insight you gain might surprise you and point you to where the real problem lies. If you find a rarely used feature(s) seriously consider removing them. You may be surprised how one such feature can bottleneck the system.

Removing a feature that is causing a table lock or extreme CPU spikes can free up those resources allowing the rest of the site to flow and  with the usage data you can assess how valuable that feature really is. If you can’t decide one way or the other, ask your users. At the end of the day you may need that feature but your users can live without it for increased performance while you work on a better long term solution.

Don’t be afraid of throwing that hard work in trash. While it may seem like your app is moving backwards what you are really doing is improving your user experience, code and performance maintainability and doing so with the right combination of data to back up this decision.

31
Jul 2011
AUTHORMike Benner
CATEGORY

Data, General, NoSQL

COMMENTSNo Comments

What Exactly Does Big Data Mean?

Every time I get into a discussion with someone on the topic of “Big Data” it seems to diverge into one of a handful of subtopics.  Whether it is what size is considered big data, what technology must you be using to be considered big data and what problem are you trying to do with your data.  These are all great buzz worthy topics but does it matter what technology you are using or truly how big your data is?

When it comes to the size of your data everyone will (and should) have a different definition of big.  Really what makes data big is dependent on the resources you have available to manage that data.  With this in mind Walmart or Facebook and their data make what I deal with tiny.  Does that mean I don’t experience similar challenges to them?  No.  While their  task is far larger and greater scope than mine, they also have deeper wallets, more personel and greater technology resources.  So just because you aren’t dealing in Petabytes doesn’t mean you are not dealing with big data.

NoSQL and Hadoop are a requirement for being in the big data space right?  I mean after all that is why those tools where invented and if you aren’t using them to solve your problems then you clearly haven’t entered into the big data space.  While those tools can be useful (or even detrimental) they are not required to be in the big data space.  MySQL, Postgres and the others have been around for ever and people have been using them to solve their data problems without the other tools.  Hell, I know of one company that deals with what I would call Gigantic Data and does it all in flat files (although I guess those are the original NoSQL solution).  It is not the tools you are using, it is how you are using them and what you are trying to accomplish.

Which brings us to what I think the real problem is, what are you trying to solve?  Or better yet what question are you trying to answer.  If you don’t really know, then you are data warehousing and most likely dealing with archiving big data sets but not really in what I would call the big data space.  This is where I think things get muddy for most.  To me all the new tools at our disposal and the fact that storage is so damn inexpensive is causing us to archive everything and we don’t know why, but we have a gut instinct that it will be worth something someday or it holds the answer to some unknown question.  Both of those maybe true, but the data isn’t the piece that is truly valuable, it is the unknown question that will bring value.

“Big Data” is asking your data set for an answer to a question and getting that answer as quickly as possible.

22
Jul 2011
AUTHORMike Benner
COMMENTSNo Comments

OSCON Data 2011

I am going to be up in Portland for the OSCON Data tracks this upcoming week. Are you going?


OSCON Data 2011

09
May 2011
AUTHORMike Benner
CATEGORY

General

COMMENTSNo Comments

Using memcached: How to scale

Using memcached: How to scaleUsing memcached: How to scale by Finsel
My rating: 4 of 5 stars

Good book for helping catch those mistakes you could make implementing memcached for the first time. This is definitely worth a read if you are thinking of implementing any NVP cache system. The lessons conveyed in this Using memcached can be applied to not only memcached, but Redis, Riak, etc.

View all my reviews