I’m working on a Python project that needs to run as root in order to work properly. Previously I’ve just run the whole PyCharm IDE as root, but this has some down-sides, and I think I have a better approach now.
Learning about cryptography can be discouraging. You get so bombarded by “don’t invent your own”, “you’re doing it wrong”, and “even really smart people screw this up” that you wonder why you even bother to try. For me, the answer is because if you don’t learn it, someone who knows even less than you will end up implementing it (badly, but with confidence) on your project. So despite being fraught with peril (because I am not an Expert), I’ll share a little about the concepts of public key cryptography in a form that has been helpful to me.
Software is kind of cool in that you can write programs that verify that your other programs work correctly. These testing “meta programs” tend to get short-shrift though, because it’s not like the code is actually part of the shipping product. So who cares about cleanliness or good style or all that stuff?
I’d propose that a good rule (or at least guideline) for test code is:
Treat it like production code.
That means things like:
- Keep methods short
- Name things appropriately
- Avoid duplication (like copying and pasting big chunks of code)
- Add comments if the code isn’t self-explanatory for some reason
- Readability counts
The main justification is that the tests need to be correct and maintainable too, and all the things we’ve learned about good code contribute to those objectives. A large body of unmaintainable tests starts to become a liability rather than an asset. The one concession I’d make is that tests pretty much never require their own tests, because then you have a problem of infinite recursion.
APIs are user interfaces for programmers. I came across a function recently that had a couple of user interface issues that serves as a great example of this principle.
The function is internal to an HTTP client class, taking care of the common logic independent of the HTTP method, and the signature looked something like this:
def _send(method, content_type): ...
This looks pretty straight-forward to programmers used to HTTP, but the parameters are deceiving.
First of all,
content_type is not actually a full content-type, but rather only the content subtype. In other words, it takes whatever you give it and prepends
'application/', so rather than passing in a content-type like
'application/json', you’d just pass in
'json'. That’s not a huge deal, but a programmer’s user experience would be that much smoother if the parameter name were
The other parameter,
method, you might expect to be a string indicating the HTTP method for the request. That would be wrong. It is actually a function object from the Python requests library, such as
requests.post, that will be used to actually send the request off. Maybe
method_function would be a better name.
Sure, sometimes you need a little context to understand that your first thought of what something is isn’t correct, and that the design choice is actually reasonable, but little details can prevent bugs, like slightly more precise parameter names.
As an example, the function above had a line like this in the middle of it, written by one of the developers who was reasonably familiar with the project:
if method == 'GET' or method == 'POST': ...
While hooking up SQLAlchemy for a web project, I kept running into attribute errors for ‘session_registry’. I figured I was doing something wrong, so I thought I’d just walk through the tutorial as a sanity check.
I was, apparently, insane.
>>> u = session.query(User).filter_by(name='ed').first() Traceback (most recent call last): File "", line 1, in File "C:\dev\misc\SundanceAPI\sapi_env\lib\site-packages\sqlalchemy\orm\query.py", line 2282, in first ret = list(self[0:1]) File "C:\dev\misc\SundanceAPI\sapi_env\lib\site-packages\sqlalchemy\orm\query.py", line 2149, in __getitem__ return list(res) File "C:\dev\misc\SundanceAPI\sapi_env\lib\site-packages\sqlalchemy\orm\query.py", line 2349, in __iter__ context = self._compile_context() File "C:\dev\misc\SundanceAPI\sapi_env\lib\site-packages\sqlalchemy\orm\query.py", line 2702, in _compile_context context = QueryContext(self) File "C:\dev\misc\SundanceAPI\sapi_env\lib\site-packages\sqlalchemy\orm\query.py", line 3247, in __init__ self.session = query.session_registry AttributeError: 'Query' object has no attribute 'session_registry'
Tom Christensen, after joining me in some head scratching, noticed that I was on the 64-bit build of Python, and wondered if there could be anything strange with that.
Googling, we found:
TODO: Still valid? The ez_setup.py script currently(?) installs a broken version of setuptools on 64-bit Windows systems.
TODO: Still valid? To work around this you need to manually download SQLAlchemy and extract it. When extracted you need to copy the <top_dir>/lib/sqlalchemy directory into your Python<version>\Lib\site-packages directory.
Sure enough, following that work-around got rid of the errors. So thanks, openlp guys. In the end, I swapped my virtual environment out for the 32-bit variety. It feels safer there.
About a year ago, I set out to get a solid working understanding of Python packaging, resulting in a lengthy blog post of what I learned. There have been some significant course changes over that year such that my original post is wrong in several regards.
Python 3.3 includes a built-in equivalent to the popular virtualenv tool for creating isolated Python environments. One difference with the built-in venv is that it doesn’t automatically install pip as virtualenv does. While I can understand the decision (given Python’s volatile packaging situation), it isn’t very convenient.
Here’s a Windows batch file that will remedy that, using the latest recommended way of getting pip. It depends on having curl in your path somewhere. Use it like you would virtualenv or venv, namely, by providing the path to where you’d like to create your new virtual environment.
@echo off REM Python 3.3 pyvenv including pip. REM See https://python-packaging-user-guide.readthedocs.org/en/latest/setup.html if "%1" == "" goto :error python -m venv %1 call %1\Scripts\activate curl -O https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py python ez_setup.py curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py python get-pip.py del ez_setup.py del get-pip.py del setuptools-*.tar.gz goto :EOF :error echo Specify the path to the virtual environment you want to create. echo venv my_env