May 042014
 

0.1.3, 0.1.4, 0.1.5 – The Lost Versions

2014-05-05, I am updating this post to give the short answer: use python setup.py register to update your meta-data on pypi. It can be run repeatedly and will modify the meta-data on pypi for the distribution.

and now the original blog post…

During the first launch of YamJam, I went through a series of releases because of rendering issues of README.rst on pypi. WAT!, I say to myself and a few other choice words. I had just created 6 pages of rst, that compiled just fine for my sphinx generated documentation. I had cut and pasted the top portion of my index.rst with some text edits — what can be going on? I uploaded my package with twine – no errors. pypi was seeming to say, everything is peachy, then turning its back and mumbling FU. (Things should not fail silently.) I pulled up my shiny new package on pypi and was met with unseemly, unformatted text instead of a spiffy display.

I google for answers, I review other dists in pypi, pulling up their repos and reading through their readmes — Aha! I say to myself, I don’t see anyone else using an :alt: on their build status badge. I remove mine, go through the release procedures again, I upload and was slapped down yet again.

I grow angry and frustrated, I recheck my readme, I google for issues related. I find mention of “run the docutils command on it”, but no mention of what command. I review even more dists on pypi, seeing other broken readme renderings. I am unsure of what to do. I think — “It must be my .. code:: python, I see lots of :: , I rip out mine and replace with ::. I try again and FAIL.

I reread the rst, I put it through sphinx just by itself and see a warning about duplicate references. I had `view <url1>`_ for docs and `view <url2>`_ . One of those little edits, I mentioned earlier. You think, looks like an href, smells like an href — but no, it is different. That did it, 3 revisions later, I’m happy with the display of my readme on pypi.

It was a day later, while enhancing my release and CI scripts that I found the python setup.py check and python setup.py check --restructuredtext commands. I tested it locally, and sure enough, it would “warn” but not set an return code, so my release script couldn’t detect the failure. I figure, that messing up your pypi page, should equal FAIL not warn. Ok, so I’ll submit a patch to make it ERROR and set a non-Zero return code. I find the code repos and start reading through the code and discover an option I hadn’t seen. -s, strict. That will cause it to FAIL and set a non-Zero return code. So off to the CI script I go, I add the strict option and the test passes when it should have failed. If you don’t have docutils installed, and I didn’t on my CI, it just returns the same response as if it had passed. FACE PALM. pip install docutils

As it stands now, I can detect rst that will cause pypi to fail silently so I am good in that regard, and you should be to, now that you have read this.

Sidebar: why dosen’t setup.py upload and twine upload automatically run the checks supplied by setup.py in their strictest mode? Fail early, the cost is less

Which brings me to unnecessary binding. Why is the description on pypi so tightly and unnecessarily bound to a distribution release. Forcing a new release to fix render problems and typos? Pypi will let us upload packages but not let us edit the description in a web interface WITHOUT having to do an entire re-release? Use the readme as a starting point, let us edit without re-releasing.

Alex G, if you happen to read this — please make this possible on warehouse. Mr Gaynor, Tear Down this binding! Also, Give us download stats, with as much info as possible so we can weed out mirror requests.

Anyone who is thinking, “I’m going to rewrite pip because of X, DON’T” We have had too many installers, to many distutils and setuptools. setup.py is a cacophony of knobs, buttons, dials, many fighting each other. “There should be one– and preferably only one –obvious way to do it.” Unless it has to do with setup.py, then it should be as confusing as possible. There needs to be a pypi/setup.py BDFaW (Benevolent Dictator For a While). If the new solution doesn’t solve the current problem, or creates new problems, it is not a solution, it is just change for the sake of change. Eggs and wheels – harumph, I say.

I enjoy writing code in Python, I endure creating a release.

The world can and should be a better place.

  4 Responses to “pypi vs. README.rst – a tale of frustration and unnecessary binding”

  1. Mikhail emailed me to say that you can modify meta-data by re-running the python setup.py register

    Further research finds that after you log in to PyPi you can edit your meta-data via the web.

    However, the function “edit” is hidden until you log in. I’ve never logged in to pypi before so I didn’t realize that it allowed this functionality. It is hidden until you log in. There is no reason listed on the project page to login.

  2. Thank you for this information about the “check” and “strict” and “–restructuredtext” options — I had never heard of them before!

    But please note that you can edit the description of a current or old release as many times as you want. The command “python setup.py register” will re-post whatever your current README says, so that you can edit the README and re-register a particular version at any future point you want. I often, after a release, go through several “register” cycles before I am happy with how the README looks when posted to PyPI.

  3. And in response to your comment about Mikhail: I strong advise against ever using the inline editing functionality of PyPI, except for doing things like deleting releases with critical security bugs if you think you must. Making a change directly leaves you without any formal trace of the change you made in your project version control, which to me is a critical blocker. Instead, I make the change in version control — even if I need to make a little maintenance branch on an old version to do so — and then always use register to push it from there, so I keep my own copy of the new metadata.

  4. You should use https://testpypi.python.org/pypi for testing and only upload to the actual pypi if everything works well. Also see https://hynek.me/articles/sharing-your-labor-of-love-pypi-quick-and-dirty/

Sorry, the comment form is closed at this time.