0.1.3, 0.1.4, 0.1.5 – The Lost Versions

2014-05-05, I am updating this post to give the short answer: use python setup.py register to update your meta-data on pypi. It can be run repeatedly and will modify the meta-data on pypi for the distribution.

and now the original blog post…

During the first launch of YamJam, I went through a series of releases because of rendering issues of README.rst on pypi. WAT!, I say to myself and a few other choice words. I had just created 6 pages of rst, that compiled just fine for my sphinx generated documentation. I had cut and pasted the top portion of my index.rst with some text edits — what can be going on? I uploaded my package with twine – no errors. pypi was seeming to say, everything is peachy, then turning its back and mumbling FU. (Things should not fail silently.) I pulled up my shiny new package on pypi and was met with unseemly, unformatted text instead of a spiffy display.

I google for answers, I review other dists in pypi, pulling up their repos and reading through their readmes — Aha! I say to myself, I don’t see anyone else using an :alt: on their build status badge. I remove mine, go through the release procedures again, I upload and was slapped down yet again.

I grow angry and frustrated, I recheck my readme, I google for issues related. I find mention of “run the docutils command on it”, but no mention of what command. I review even more dists on pypi, seeing other broken readme renderings. I am unsure of what to do. I think — “It must be my .. code:: python, I see lots of :: , I rip out mine and replace with ::. I try again and FAIL.

I reread the rst, I put it through sphinx just by itself and see a warning about duplicate references. I had view <url1>_ for docs and view <url2>_ . One of those little edits, I mentioned earlier. You think, looks like an href, smells like an href — but no, it is different. That did it, 3 revisions later, I’m happy with the display of my readme on pypi.

It was a day later, while enhancing my release and CI scripts that I found the python setup.py check and python setup.py check --restructuredtext commands. I tested it locally, and sure enough, it would “warn” but not set an return code, so my release script couldn’t detect the failure. I figure, that messing up your pypi page, should equal FAIL not warn. Ok, so I’ll submit a patch to make it ERROR and set a non-Zero return code. I find the code repos and start reading through the code and discover an option I hadn’t seen. -s, strict. That will cause it to FAIL and set a non-Zero return code. So off to the CI script I go, I add the strict option and the test passes when it should have failed. If you don’t have docutils installed, and I didn’t on my CI, it just returns the same response as if it had passed. FACE PALM. pip install docutils

As it stands now, I can detect rst that will cause pypi to fail silently so I am good in that regard, and you should be to, now that you have read this.

Sidebar: why dosen’t setup.py upload and twine upload automatically run the checks supplied by setup.py in their strictest mode? Fail early, the cost is less

Which brings me to unnecessary binding. Why is the description on pypi so tightly and unnecessarily bound to a distribution release. Forcing a new release to fix render problems and typos? Pypi will let us upload packages but not let us edit the description in a web interface WITHOUT having to do an entire re-release? Use the readme as a starting point, let us edit without re-releasing.

Alex G, if you happen to read this — please make this possible on warehouse. Mr Gaynor, Tear Down this binding! Also, Give us download stats, with as much info as possible so we can weed out mirror requests.

Anyone who is thinking, “I’m going to rewrite pip because of X, DON’T” We have had too many installers, to many distutils and setuptools. setup.py is a cacophony of knobs, buttons, dials, many fighting each other. “There should be one– and preferably only one –obvious way to do it.” Unless it has to do with setup.py, then it should be as confusing as possible. There needs to be a pypi/setup.py BDFaW (Benevolent Dictator For a While). If the new solution doesn’t solve the current problem, or creates new problems, it is not a solution, it is just change for the sake of change. Eggs and wheels – harumph, I say.

I enjoy writing code in Python, I endure creating a release.

The world can and should be a better place.

Since I didn’t get to attend this year, again, I’ve been watching the PyCon videos (thanks pyvideo.org and PyCon!) Out of all the video’s, Carl Meyer’s talk on “Set your code free…”, struck a nerve.

I have a project, YamJam, that I have been using since 2009. The main idea is a framework that allows you to factor out sensitive data from your code before you upload to a 3rd party repos. I’ve got internal and external projects that have been using it for quite a while and it makes these refactorings a breeze. I have other open source projects that get a lot more attention but yet have a niche audience, so I am experimenting to see if it is the lack of documentation and being a conforming pypi dist that is limiting it’s appeal. Yamjam should be popular, because it can scratch an itch caused by Djano. That itch being, “What do you do with settings.py?” It’s got lots of sensitive data that shouldn’t be checked into to a repos but it also has a lot of code that should. It also makes deployment between dev, staging and production easy to do with a checkout.

To that end, I’ve been creating a proper and complete test suite (was doctests) using py.test and tox, Continuous Integration with drone.io, documentation via readthedocs.org and sphinx, spiffing up the setup and dist with the latest distutils and uploading with twine instead of setup.py upload.

Through this whole process, I’ve had a lot of new experiences that I am going to be blogging about in the upcoming weeks. Things I like, really like, things that are annoying and some things that are counter to my way of thinking. During this process I’ve also been filing bug reports when I’ve encountered them and sending out feedback for improvements along the way.

After moving my code from subversion on google code to mercurial on bitbucket (hg convert), I started looking for a CI service to use. Off the bat, I looked at Travis-ci, but unfortunately, travis is a github snob. If you are not hosting your code on github or mirroring off of github then travis-ci is not an option. Some google searching showed that pylint (a tool I like very much and use) moved from internal tools to bitbucket and http://drone.io/ . So off to drone I go.

http://drone.io/ took about 10 minutes from signing in via my bitbucket account to running my first integration. When I learned that drone.io allows you to view the settings for other open source projects build environments, I had python 27, 32, 33 and 34 tests running via tox less than an hour later. What higher praise could I give a service than saying from 0 to testing in 10 minutes? I really, really like drone.io and recommend that you check them out. Unlike Travis-ci, drone.io supports bitbucket, github and google code. Options, I like. It is the same reason I prefer bitbucket to github. Bitbucket supports mercurial and git. I use both dvcs systems. I prefer mercurial. I like that bitbucket allows me to make the choice. Choice makes me happy. Check out my setup on drone.

More to follow.

Noah Gift over on O’Reilly OnLamp Blog has an article on building a greedy coin changer. That is, given a value, say 71 cents, calculate the fewest coins needed to make the amount. He had listed a number of solutions, but I felt I could do it a bit more pythonic. ðŸ˜‰

#!/usr/bin/env python
"""implement a greedy coin changer, returning the
fewest coins to make the change requested."""
#coin_list can be expanded to include silver dollars
# and 50 cent pieces by just expanding the coin list
# to [100,50,25,10,5,1] the reulting answer
#structure will modify itself to reflect

coin_list = [25,10,5,1]
change_requested = .71
remaining = change_requested * 100
change_returned = []    #result structure

for coin in coin_list:
num_coins,remaining =  divmod(remaining,coin)
change_returned.append(int(num_coins))

print change_returned
print remaining


The benefits of this version, are no conditional logic is needed, the coin structure can be modified and the answer will modify itself accordingly.