John Lennon vs. Paul McCartney: A Quantitative Analysis

A friendly debate has long raged between Beatles fans on the subject of which of the Fab Four was the better songwriter, John Lennon or Paul McCartney.  I decided to try to find an objective answer this question by using quantitative techniques.

Background

The Beatles, of course, had four members, John Lennon, and Paul McCartney, George Harrison, and Ringo Starr.  From 1963 to 1970 they released 210 songs, of which 185 were released on their 14 studio albums, with the balance (2010 – 185 = 25) released but not as part of an album.

Of the four members, Lennon and McCartney were the most prolific songwriters.  The two agreed to officially share credit as “Lennon/McCartney” on all songs either of them authored as part of the Beatles, but through various secondary sources such as interviews we know that many of the songs were not collaborations and instead were written principally by one or the other.  In fact, across the 210 songs in the Beatles discography, 72 were written principally by Lennon and 70 principally by McCartney.  A further 18 songs attributed to Lennon/McCartney (e.g. “Wait”) were written in genuine collaboration and not principally by one or the other.

I obtained the above information on authorship of Beatles songs from fan Per Myrsten’s Beatles Discography.

Approach

I decided to cross-reference the song authors with a ranking of the Beatles’ songs.  Rankings are available from the crowd-sourced ranking site ranker.com as well as from professional critics at Rolling Stone Magazine.  Of the two I decided to use the ranker.com rankings.

To scrape the rankings from the webpage I wrote a Python script that uses a scraping library from Leonard Richardson called Beautiful Soup.  Click here for the source code of my scraping code.

I then set up an Excel spreadsheet with the rankings and the author of the song on one tab, and the cumulative songs for each other and rank of the second.  The formula for Column B is =COUNTIF($I:$I, "<="&$A3) and the formula for column D is =B3/SUM($B3:$C3) (at least on the first row).

Here are the first 20 rows:

	Cumulative Songs		Proportion of Cumulative Songs
Top X	Lennon	McCartney	Lennon	McCartney
1	0	1	0%	100%
2	0	1	0%	100%
3	1	1	50%	50%
4	1	2	33%	67%
5	1	2	33%	67%
6	1	3	25%	75%
7	2	3	40%	60%
8	2	3	40%	60%
9	2	4	33%	67%
10	2	4	33%	67%
11	3	4	43%	57%
12	3	4	43%	57%
13	3	4	43%	57%
14	4	4	50%	50%
15	5	4	56%	44%
16	6	4	60%	40%
17	7	4	64%	36%
18	8	4	67%	33%
19	8	5	62%	38%
20	9	5	64%	36%

Then I charted columns D and E:

Conclusions

Notice that since McCartney wrote the top-ranked Lennon or McCartney song (Hey Jude) at the #1 rank he has 100% of the songs.  The second-ranked song is Strawberry Fields Forever, by Lennon, so of the top 2 songs, so McCartney’s share has dropped to 50%.

McCartney actually has 67% of the top 10 McCartney / Lennon songs.  But by the time we get to the top 20 songs, it is Lennon who is ahead, with 64% of the songs.  Lennon appears to have authored more of the bottom-ranked songs, so McCartney did author a higher-than-50% share of the middle-ranked songs.

These figures leave the final answer of who was better undecided.  It appears that both men were quite evenly matched.  This seems to validate the conclusion of many fans that both men were very nearly equally talented.

It would be interesting to perform this same analysis using the Rolling Stone rankings.  If anyone does this, please let me know in the comments.

History is written by the historians

History doesn't get written by the victors, it gets written by the historians.  So it is possible to significantly influence people's understanding of history simply  by being the one who records it.  I often am probably the only one who has written of a party attended by people, and so 50 years from now my record will be the only thing anyone knows of the event.  In a sense by caring enough to write it down, I have elevated the historical importance of my perspective on the event, since mine is the only perspective people will see.

History tends to be thought of as the chronicle of very large, important events, like the battle of hastings or world war II.  But for individual people, the history of their own lives, their immediate ancestors, or the neighbourhood in which they live can be even more fascinating and also useful to future planning.

How Some Groups Can Fail

It's fascinating how groups of people who are all friendly people, who all want the same thing and don't even disagree about what is to be accomplished, can end up accomplishing nothing and even bitter and angry at one another!  In my experience, this happens when people err in one of two ways:

1.       One or more individuals do not respect the boundaries of their commitments, committing and then not following through on that commitment.  Notice that is perfectly okay to not commit to something, if you are busy, etc., but the problem comes when a commitment to have something done is made and then the deadline passes without any acknowledgement or conversation about fixing it.

2.       There are unspoken differences in assumptions about who is to be working on what.

Now, (1) is likely a general problem in that individual's life.  The harder, more complex problem is (2), since it is a problem of group dynamics.  What can allow (2) to happen is if one person makes themselves unavailable, i.e. will not respond to calls or emails.

The job of a leader is essentially to resolve these two issues.  If a group lacks a strong leader, these two issues can fester and cause bitterness and resentment, as well as the failure of the original goal of the group.

Plugged Pipes in The Productivity Engine

If you have a work pipeline where you just put things in but never or very rarely pull things out, it is a broken workflow that may be stopping up your whole productivity engine.

For example, I used to "star" items I wanted to follow up on in Gmail, but I never systematically tried to review those stars afterwards!  It was a "write-only" system.  Only when I linked Gmail to Outlook and started using the "tasks" view did I start systematically taking care of the backlog of hundreds of starred messages.  My friends were sometimes suprised to hear back from me about years-ago messages I never got back to them on.

Another example, that is in fact still broken for me, is my "to read" pile.  I have for years maintained a list of links and books and movies I found that I felt would be good to read or watch later.  But whenever I actually get free time to read, I always go online, perhaps on Facebook, and randomly collect a brand new link to click on.  Clearly this process is broken!  I should think about fixing it.

Implementing GTD and Inbox Zero using Outlook 2013 with a Gmail account

I’ve used Gmail since 2003, and I love its slick interface, which remains superior in my opinion to any other web mail out there.  Like any web mail, it’s available anywhere I am, on any computer.  I can even “star” any messages that are either next actions or items I’m waiting on someone else for.  It also has Google’s powerful and fast search.

However, at work I’ve always used Outlook.  Outlook also has a slick interface, and its equivalent of “star”, the “flag for follow up”, is even more powerful because it can be supplemented by colour categories, which I use to represent the context where the action needs to be done: “At Computer”, “Errand”, etc.:

(To get categories nicely set up like this, follow the advice from Outlook Guru Diane Poremsky at http://www.slipstick.com/outlook/outlook-categories-flags-and-imap-accounts/)

I also find Outlook very satisfying because I can drag messages to appropriate folders, and the folder hierarchy is much easier to manage than in Gmail.

Furthermore, there is a separate page called “tasks” where I can view all my flagged messages at once, grouped either by folder or by category.  This makes for a powerful way to implement GTD and Inbox Zero.

So I want the best of both worlds: while at my home office I want the power and features of Outlook, but I want to retain the option of searching and viewing my messages through the Gmail web interface.

You would think this would be simple: just click File -> Add Account and then give the information for Gmail.  Unfortunately, there are some details that if you get wrong, will cause a huge amount of frustration.

I set mine up that way and it worked for a while, but it would intermittently not work.  One day I had to spend a day working with Outlook while offline.  I spent the whole day checking off flags, and moving emails around into various folders.  After that day the sync no longer worked.

The Issue

As I found, the Outlook to Gmail IMAP interface can be painfully slow (as described in this video), and in fact apparently can stop working completely, unless you have certain settings correct.  You can easily get stuck on "synchronizing subscribed folders" during the Send/Receive step, and Outlook will either take forever or never complete the Send/Receive.

This is the IMAP issue I am referring to:

http://answers.microsoft.com/en-us/office/forum/officeversion_other-outlook/synchronizing-subscribed-folders-issue/ec6835a6-cf06-4f9f-9432-37af831631c4

I paid $100 twice for Microsoft Premium Support, but they were unable to fix the issue.

I also considered just migrating all my Gmail messages into Outlook.com (formerly Hotmail), to take advantage of the fact that this webmail uses the Microsoft-Proprietary ActiveSync protocol, which might not have had the issues that the IMAP protocol has:

http://www.labnol.org/internet/import-gmail-into-outlook/24518/

http://www.pcworld.com/article/2011519/how-to-move-from-gmail-to-outlook-com.html

The old tool used for this purpose was called “TrueSwitch” but it is no longer supported, as even Google itself points out:

http://www.dataliberation.org/google/gmail

Another option was to use the Windows Live Essentials to import/export mail messages.

Fortunately I found the solution described below before taking the drastic step of migrating away from Gmail.

The Solution

The only solution that I found was:

(1)    Create New Profile

If your Outlook sync is not working, I recommend you first create a totally new profile and download the messages from the Gmail server again.  I know this is a giant pain if you have gigabytes of data, but on a strong connection it should only take a couple of hours, and for me it was the only way to ensure that the other steps below actually worked.

(2)    Modify “Send/Receive Groups” settings

Run Outlook /profiles and select your new profile.  Then Press Ctrl-Alt-S to visit “Send/Receive Groups”.  Alternatively you can access it from the Ribbon here:

I clicked “Edit” and followed the instructions from vishalkarpe in http://answers.microsoft.com/en-us/office/forum/officeversion_other-outlook/synchronizing-subscribed-folders-issue/ec6835a6-cf06-4f9f-9432-37af831631c4

1) Un-check the "Get folder unread count for subscribed folders"

2) Under Received mail items, check the option "User the custom behavior defined below"

3) In the window below that displays your Gmail folders just select inbox and leave the others unchecked

4) Click ok and then exit out of options and perform a send/receive to test if it is now working

(This advice is also contained in the following links:

http://www.youtube.com/watch?v=-6wXTP1AIq8

http://www.centennialarts.com/support/2012/05/11/outlook-hangs-when-synchronizing-subscribed-folders/)

If all goes according to plan, your Gmail inbox will also be empty!  All messages are now classified in folders.

As a test, check if when you star an email in Gmail, it shows up flagged after the sync in Outlook, and vice versa!

N.B.: If you ever feel like you might have lost an email across both systems, it might have for some reason ended up in under the “All Mail” label only.  Check there.

 Backing up the .ost File

I never had to worry about losing my email before, since it was all stored on the cloud at Google’s servers.  But the offline copy of the emails that Outlook maintains stores some extra information that is not copied by the IMAP protocol back to Google’s servers.  In particular, views, rules, and categories are not stored.  So it would behoove you to back up that offline copy.  I’ve done my backup via DropBox.

To do this I first found the location of the offline copy.  I went to File -> Account Settings -> Data Files -> Open File Location…

Normally this file location wouldn’t be within your DropBox folder, so you’ll have to use a trick to automatically create an “Echo” of this folder within your DropBox folder.  Follow the instructions here to use a free Microsoft tool called SyncToy:

http://www.dropboxwiki.com/tips-and-tricks/sync-other-folders

Schedule to run once a day:

http://www.labnol.org/software/backup-files-dropbox/

I hope this helps someone out there!