Estimating a population mean from a sample in python

Least Squares Linear Regression with Python

Some python examples from UCSD stats 650, Fall 2014

Fun with sinuses

20121231-113712.jpg

Caronas Sandwich Shop

A new series where I review one of my former jobs. Names have been changed to protect the guilty.

In 1995 I took my second trip cross country, and upon arrival at my half-brother’s house in San Diego, broke up with my hippy, vegan, dreadlocked girlfriend, Caitlin. She and I had come to the point in our relationship where constant bickering over life’s inconsequential minutiae had become surprisingly less entertaining than expected.

Alone, immature, and crashing at my then thirty-something brother’s pad with his new wife proved inconvenient for both of us, namely him, so it was on me to get a jobby job and find a place of my own. This being  San Diego’s mid-90’s salad days, I was quickly able to find a job at a Mission Hills sub shop,  the most menial possible job possible within a bike ride of my temporary home base in Normal Heights.

My job was clear: I’d do all the sh*t work that no one else would be bothered to do. In food service this means lots of cleaning, from grease traps to ovens, and then for a treat,  the lowest end of food prep, i.e shredding lettuce, cooking bacon, and slicing tomatoes, i.e. anything that requires zero cooking skills.

The boss was Steve Carona.   His namesake eatery was a typical non-descript sandwich joint in that one spot that has turned over 10 times on an otherwise nice little restaurant row.  Steve had a standing order for at least a few dozen subs a day on to the base, which basically kept the place running while he squandered any meager profit either on one of his many ex-wives and mistresses or more likely, up his nose.

Steve was a massive pain in the ass, but was rarely there.  My day started with Howard Stern on the radio while sweeping the leaves from a nice, shady upstairs eating area.  Then it was time to prep for the lunch rush.   My least favorite job was in the afternoon: protecting the private, shared parking lot from non-authorized-parkers, something which basically involved me counting minutes while staring into space. Occasionally, I had to clean the grease trap, or cook 50lbs of bacon at a time, and inhaling bacon grease vapor is not something I would recommend to anyone.

Honestly, it wasn’t a bad job. It was one of those jobs that’s just exactly what you’d expect.   The paychecks cashed and the people were nice.  I made friends with the  lead cook and we’d sometimes hang out at her under-furnished North Park apartment and drink beer.  Eventually I saved up enough to get out of my brother’s house.  I think I quit because I found a better job somewhere else, and no one really cared one way or the other.  Carona’s closed a few years later.

15 years later,  I find myself picking up a pizza pie from my favorite local place, and who’s standing outside, greeting customers and answering the phone?   None other than Steve Carona.  I shake his hand and tell him that I worked for him a dozen years ago.  His interest piqued, but clearly having no idea who I was, he responds like he was running the place:  “Oh yeah?  Well, I’m over here now”.   That was pretty much about what I’d expected.

Facebook isn’t really your friend

The Social Graph is Neither (Pinboard Blog).

You might almost think that the whole scheme had been cooked up by a bunch of hyperintelligent but hopelessly socially naive people, and you would not be wrong. Asking computer nerds to design social software is a little bit like hiring a Mormon bartender. Our industry abounds in people for whom social interaction has always been more of a puzzle to be reverse-engineered than a good time to be had, and the result is these vaguely Martian protocols.

Nailed it.

The Corporate IT Hierarchy

  • Management
  • Database Admins
  • Project Managers
  • Software Engineers
  • Business Analysts
  • Configuration Management
  • Network admins
  • Sysadmins
  • Quality Assurance
  • Helpdesk

Framistan

“The Enterprise computer system is controlled by three primary main processing cores cross linked with a redundant melacortz ramistat and fourteen kiloquad interface modules. The core elements are based on FTL  nanoprocessor units arranged into twenty-five bilateral kelilactirals with twenty of those units being slaved to the central heisenfram terminal…. Now this is the isopalavial interface which controls the main firomactal drive unit… The ramistat kiloquad capacity is a function of the square root of the intermix ratio times  the sum  of the plasma injector quotient…”

– Cmdr. Riker

My Hometown – Bruce Springsteen

I was eight years old and running with a dime in my hand
Into the bus stop to pick up a paper for my old man
Id sit on his lap in that big old buick and steer as we drove through town
Hed tousle my hair and say son take a good look around
This is your hometown, this is your hometown
This is your hometown, this is your hometown

In `65 tension was running high at my high school
There was a lot of fights between the black and white
There was nothing you could do
Two cars at a light on a saturday night in the back seat there was a gun
Words were passed in a shotgun blast
Troubled times had come to my hometown
My hometown, my hometown, my hometown

Now main streets whitewashed windows and vacant stores
Seems like there aint nobody wants to come down here no more
They’re closing down the textile mill across the railroad tracks
Foreman says these jobs are going boys and they aint coming back to
Your hometown, your hometown, your hometown, your hometown

Last night me and kate we laid in bed talking about getting out
Packing up our bags maybe heading south
Im thirty-five we got a boy of our own now
Last night I sat him up behind the wheel and said son take a good
Look around
This is your hometown

Entity Framework: the OMG ORM?

ORM (Object-Relational Mapping) technology has been around a while now, and has moved from niche to mainstream along with code generation, unit testing, and agile development. ORM, in a nutshell, allows a developer to link object models to relational database schemas, eliminating the impedance mismatch inherent between relational databases and object-oriented programming languages. ORM can effectively eliminate the tedium and clumsiness of writing low-value SQL queries and can provide some pretty significant improvements in the deployment process. I don’t want to extoll the virtues of ORM too much here, this article will assume you are familiar with the concepts. Popular tools like Ruby On Rails’ ActiveRecord, and Java’s Hibernate are widely used implementations that can give you more background on the technology as well.

Microsoft has recently released Entity Framework 1.0 (EF) in Visual Studio 2008 SP1, it’s first, albeit very late, entry to the ORM world. I have several years experience with open-source NHibernate, a .NET port of Hibernate and the standard ORM technology for the .NET platform. While not perfect, NHibernate has been an essential workhorse in my toolkit, and one that I have leveraged to great success in both simple and complicated projects. I decided to take a look into EF to see how it compared to NHibernate.

The biggest and most obvious difference is that EF has a very slick UI built right into Visual Studio.NET. This UI allows you to use a wizard-style interface to select database objects and generate the entity model with a few clicks. Changes to friendly names and collections are easily done with a simple click and rename.Foreign Key relationships are automatically translated into object collections with no coding necessary. Under the hood this is all code generation – in fact an incredibly complicated set of C# classes that cannot and must not be modified by hand.

NHibernate, by contrast, relies on the much-maligned XML configuration file for it’s mappings. These files are well-known for being antagonistic to new users, but with some experience, are reasonably easy to get down. Visual mapping tools and codegens do exist (I have used a customized MyGeneration template for years) although they are not nearly as polished as the EF designer in Visual Studio 2008.

Once I had taken a spin through the EF designer and generated some classes, it was time to perform some basic CRUD operations against my database. EF does not provide pre-built CRUD code, instead it relies on Microsoft’s new LINQ-to-Entities syntax to allow a C#-native, SQL-style syntax for querying the entity model. Writing code to get an object by it’s ID via LINQ is very simple, and writing it in C# with full Intellisense support and syntax highlighting is a nice feature.

NHibernate has a query language too, although not native to C#, it is essentially the same as LINQ, a pseudo-SQL language called HQL. For much day-to-day work there is not much of a difference in the query language aspect between these two ORM’s, but LINQ definitely has the advantage here. The recently released LINQ-to-Hibernate provider should close this gap considerably.

Entity Framework does a good job of hiding the database, perhaps too good of a job. While NHibernate just needs a connection string to access a database, EF uses a set of special files to store the model and it’s mappings. Managing these files and their connection strings can be troublesome when accessing the model from separate projects (i.e. a unit test project and a web app). Also, NHibernate exposes the database connection a bit more explicitly, exposing a Session object and offering numerous ways to control the session, as well as some fairly deep features such as lazy loading, caching, and various collection types which are not present in EF.

The main challenge with EF becomes apparent when you attempt to build an object model beyond the basic object-per-table paradigm the EF GUI exposes. If you want to create a rich object model with modern OO techniques of aggregation, composition, generalization and specialization, it will be very hard to do with the EF designer. Dropping out of the designer presents a level of complexity much deeper than NHibernate’s config files.

EF’s preference for table-per-entity design creates a particular issue with legacy databases, especially databases that don’t expose good key structures (Peoplesoft is a particular example), or multiple database sources. This limitation, in my opinion, is the biggest problem with EF.Most modern software consumes multiple data sources – XML files, databases, web services, file system resources. It is usually beneficial to wrap these resources into a business entity model and not expose the underlying storage. Since NHibernate is basically just a mapping, it is less intrusive into your domain objects, and is less dependent on a ‘database-first’ approach to building out the entity model.

Ultimately, it would appear that EF, while a good first step, is probably not going to unseat NHibernate as the ORM of choice in the .NET framework. While it is an acceptable solution for most basic, CRUD heavy situations (and there are lots of them), it is probably not going to be useful to the people that often use ORM – enterprise developers working across varying data sources looking to simplify their data access into a comprehensible model. It will take time, and potentially several versions, for EF to get to where it needs to be. NHibernate has the inertia, and the large pool of experienced users that is very crucial in these types of tools.

However there is no question EF has incredible potential. Questions persist about NHibernate’s future as it has stayed at v2 for many years. Stability is not a bad thing in my opinion, but Microsoft will catch up fast as they have the resources to do so.Also interesting is Microsoft’s apparent PR blunder around Entity Framework. Announcement that EF would be the data access technology of choice going forward created a backlash around the simpler and more mature LINQ-to-SQL, which had already been in the wild for some time and was being used my many developers. Ultimately we will have to wait until 2010 and .NET 4.0 to see if EF is truly going to be the tool that will wean so many developers off of sprocs and ADO.NET code.

Jacaranda Season

Jacaranda Season, originally uploaded by Atwater Village Newbie.