Creating a custom linter can be a great way to enforce coding standards and detect code smells. In this tutorial, we'll use Sylver, a source code query engine to build a custom Python linter in just a few lines of code.
Sylver's main interface is a REPL console, in which we can load the source code of our project to query it using a SQL-like query language called SYLQ. Once we'll have authored SYLQ queries expressing our linting rules, we'll be able to save them into a ruleset that can be run like a traditional linter.
Installation
If sylver --version doesn't output a version number >= 0.2.2, go to https://sylver.dev to download a fresh copy of the software.
Project setup
We'll use the following Python file to test our linting rules:
from users.models import *
from auth.models import check_password
foo = 100
O = 100.0
my_dict = {'hello': 'world'}
if my_dict.has_key('hello'):
print('It works!')
if 'hello' in my_dict:
print('It works!')
Starting the REPL
Starting the REPL is as simple as invoking the following command at the root of your project:
sylver query --files="src/**/*.py" --language=python
The REPL can be exited by pressing Ctrl+C or typing :quit at the prompt.
We can now execute SYLQ queries by typing the code of the query, followed by a ;.
For instance: to retrieve all the if statements (denoted by the node type IfStatement):
match IfStatement;
The results of the query will be formatted as follow:
$0 [IfStatement main.py:1:9-23:10]
$1 [IfStatement main.py:1:12-23:13]
The code of a given if statement can be displayed by typing :print followed by the node alias (for instance: :print $1). The parse tree can be displayed using the :print_ast command (for instance: :print_ast $1).
Rule1: wildcard imports (inspired by F403)
This rule will flag all the imports of the form from x import *.
The first step is to get familiar with the tree structure of Python's import statements, so let's print a ImportFromStatement node along with its AST:
λ> match ImportFromStatement;
$2 [ImportFromStatement main.py:1:1-27:1]
$3 [ImportFromStatement main.py:1:2-39:2]
λ> :print $2
from users.models import *
λ> :print_ast $2
ImportFromStatement {
. ● module_name: DottedName {
. . Identifier { users }
. . Identifier { models }
. }
. WildcardImport { * }
}
It appears that the faulty part of the import statement (the wildcard: *) is represented by a WildcardImport node.
So this first rule can easily be expressed in SYLQ:
match WildcardImport;
Rule2: Ambiguous variable name (inspired by E741)
This style-oriented rule will detect variables named 'l', 'I' or 'O', as these names can be confusing.
Same as before, let's analyze the tree structure of an assignment:
λ> match Assignment;
$4 [Assignment main.py:1:4-10:4]
$5 [Assignment main.py:1:5-10:5]
$6 [Assignment main.py:1:7-29:7]
λ> :print_ast $5
Assignment {
. ● left: Identifier { O }
. ● right: Float { 100.0 }
}
The variable's Identifier can be accessed through the left field of the Assignment node. We can match the Identifier's text against a regex
by using the builtin matches method:
match a@Assignment when a.left.text.matches(`^(I|O|l)$`);
Here the Assignment node is bound to a using the binding operator: @.
Rule3: has_key() is deprecated (inspired by W601)
This rule signals uses of the deprecated dictionnary has_key method.
Here is the tree representation of a call to has_key:
Call {
. ● function: Attribute {
. . ● object: Identifier { my_dict }
. . ● attribute: Identifier { has_key }
. }
. ● arguments: ArgumentList {
. . String { 'hello' }
. }
}
This query can be expressed using nested patterns, as follow:
match Call(function: Attribute(attribute: 'has_key'));
Creating the ruleset
The following ruleset uses our linting rules:
id: customRules
language: python
rules:
- id: F403
category: style
message: "wildcard import"
note: "wildcard imports are discouraged because the programmer often won’t know where an imported object is defined"
query: >
match WildcardImport
- id: E741
category: style
message: "ambiguous variable name"
note: "variables named I, O and l can be very hard to read"
query: >
match a@Assignment when a.left.text.matches(`^(I|O|l)$`)
- id: W601
category: style
message: ".has_key() is deprecated"
note: "'.has_key()' was deprecated in Python 2. It is recommended to use the 'in' operator instead"
query: >
match Call(function: Attribute(attribute: 'has_key'))
Assuming that it is stored in a file called ruleset.yaml at the root of our project, we can run it with the following command:
sylver ruleset run --files "**/*.py" --rulesets ruletset.yaml
Getting updates
For more informations about new features and/or cool SYLQ one-liners, connect with Sylver on Twitter or Discord!
