Every time there is a discussion about SemanticDiff, some people seem to be confused about how it is different from difftastic or a standard diff with whitespace ignoring enabled. So I decided to create this blog post where we go a little deeper into how the tools work and explain their differences to help you decide which one is best for your needs. The comparison is based on the current version of the tools at the time of writing, which is 0.8.8 for SemanticDiff and 0.54.0 for difftastic. You can find a summary at the end of the post.

Structural vs. Semantic Diffs

You may have heard that difftastic provides a structural diff while SemanticDiff offers a semantic diff, but you may be unaware what that actually means. So, let’s take a step back and recap how classical diffs, structural diffs, and semantic diffs work. Afterwards we take a look at a code example to showcase the difference.

Classical diffs, such as the one provided by git diff work by reading both the old and new code, and aligning those files line-by-line. Typically, algorithms like Myers Diff are used to compute a mapping of old and new lines. Lines that have no matching partner in the other file are treated as changed, whereas lines with a matching partner are considered unchanged. Such a simple approach works well if each piece of information is stored on a separate line, but that is not really the case for code. Adding line breaks between statements is enough to break the matching, often rendering this type of diff useless when code gets reformatted.

Structural diffs, as implemented in difftastic, work differently. Instead of using the text directly, they translate the code into a tree structure based on the grammar rules of the language used. These abstract syntax trees (ASTs) basically show you the parent-child relationship of all syntax constructs in your code and which tokens belong to them. For example, for the Python code def foobar(): return 1 an AST could look like this:

function_declaration:
    "def"
    identifier:
        "foobar"
    parameters:
        "("
        ")"
    body:
        return_statement:
            "return"
            integer:
                "1"

The advantage of this approach is that whitespace or line breaks that don’t affect the code logic aren’t part of the AST and therefore get filtered out. To generate a text diff, the old and new tree get aligned by matching the nodes they have in common. Nodes that have no matching partner are treated as change. The mapping of the nodes and their text positions are then used to produce the highlighted diff output.

Semantic diffs, as implemented in SemanticDiff, go one step further. They not only take into account the information about how the tokens are structured, but also try to understand the meaning of the different code constructs. While a structural diff treats 1337 and 0x539 as two distinct tokens, a semantic diff knows that they belong to the same number. This makes it possible to filter out changes that don’t change the logic of the program, such as adding optional parentheses. Internally, both types of diffs work quite similar, but a semantic diff attaches more information to the AST and has special rules and logic to detect those invariant changes.

Example: diff vs. difftastic vs. SemanticDiff

Now that we’ve explained the differences between a structural and semantic diff, let’s take a look at an example to demonstrate what that actually means in practice. For this we use a Python snippet in which a string gets extended and split across multiple lines.

When we confront the diff utility with such a change, it leaves most of the work up to the user. There are no identical lines to match and everything is marked as change. The user has to manually figure out what happened to this block of code.

-CONST_STR = "This is our test string"
+CONST_STR = (
+   "This is our"
+   " test string"
+   " but now it"
+   " got longer"
+)

Difftastic provides a more helpful output and is able to identify that the left part of the assignment hasn’t changed. It only marks the newly added and deleted tokens as modified:

new_string.py --- Python
1 CONST_STR = "This is our test string"        1 CONST_STR = (
.                                              2         "This is our"
.                                              3         " test string"
.                                              4         " but now it"
.                                              5         " got longer"
.                                              6 )

SemanticDiff can narrow down the change even more. It is aware of how string concatenation works in Python and therefore knows that splitting the original string into two literals was a no-op. SemanticDiff also understands that the parenthesis were only added due to syntactic requirements and that they don’t modify the logic. In the end, only the new part of the string gets highlighted:

Refactoring Detection

In the last section we compared how the diffing algorithms work that power difftastic and SemanticDiff and how this influences their output. However, this alone is not the only deciding factor on how useful the generated diff is. A diff tool can also assist you with understanding certain type of changes. In the following, we want to check how the tools handle two common types of refactorings.

Moved Code

Code often gets moved around when extracting logic into a separate function or to resolve issues with the order in which identifiers are defined. Let’s take a look at how the two tools handles this.

Difftastic does not support the concepts of moves and can only try to align either the moved code or the unchanged code. In our example it decides to align the moved code, so that the actually unchanged code is marked as modified:

1 proc = subprocess.run(            1 args = (
2     [                             2     [
3         "ping",                   3         "ping",
4         "-c", "10",               4         "-c", "10",
5         "-t", "50",               5         "-t", "100",
6     ],                            6     ]
7     stdout=subprocess.PIPE,       . 
8     stderr=subprocess.PIPE,       . 
9 )                                 7 )
                                    8 
                                    9 proc = subprocess.run(
                                   10     args,
                                   11     stdout=subprocess.PIPE,
                                   12     stderr=subprocess.PIPE,
                                   13 )

SemanticDiff recognizes that a part of the code was moved and displays this information to the user. It also highlights how the moved code was modified:

Repeated Changes

Moving code is only one potential type of refactoring and there are many more. One characteristic that they often have in common is that they consist of repeated changes. An example for this are renames, where the same variable name gets changed over and over again. Let’s see how the tools handle such a case:

Difftastic shows the user each modification as individual change. It is up to the user to recognize that there is a pattern, and that the variable has been renamed everywhere.

4 function list_files(dir) {                                 4 function list_files(directory) {
5     for (const file of fs.readdirSync(dir)) {              5     for (const file_name of fs.readdirSync(directory)) {
6         const file_path = path.join(dir, file);            6         const file_path = path.join(directory, file_name);
7         console.log(file_path);                            7         console.log(file_path);
8     }                                                      8     }
9 }                                                          9 }

SemanticDiff tries to detect repeated changes and is able to group renames. It highlights the two renamed variables with different colors to help the reviewer see the pattern.

Command Line vs. DevOps Integrations

To help you decide on a tool, it also makes sense to ask yourself: When and where are you typically looking at diffs / reviewing code? Ideally, the tool should be compatible with your existing workflow, so that you get your diffs exactly where you need them.

Difftastic is a command line utility that allows you to compare files or directories on your filesystem. It prints a static text diff that can be viewed in your terminal or using a text editor. Difftastic can be used as an external diff tool by other tools (such as git) through utilizing temporary files. There is no direct integration with other software or services beyond this.

SemanticDiff focuses entirely on integration with other tools. It is available as Visual Studio Code Extension and GitHub App. The VS Code extension runs locally on your machine and can diff any files supported by the virtual file system of VS Code (local files, files in git commits, …). The GitHub App is targeted at developers who like to review pull requests in their browser with full review comment integration. SemanticDiff provides an interactive diff viewer with a minimap that allows you dynamically expand the visible context.

Supported Languages

The best diff tool becomes useless if it doesn’t support the languages used in your project. So let’s take a look at how difftastic and SemanticDiff compare in terms of language support.

Difftastic offers support for a wide range of languages and data formats (50+), from popular languages such as Rust to more niche ones like Ada. There is a good chance that the languages you are interested in are listed as supported. However, there is a small catch. The parsers used to support these languages vary quite a bit in quality. For example, the C and C++ parsers don’t handle preprocessor directives correctly nor can they handle ambiguous statements properly. This can result in parser errors or incorrectly parsed code. Problems like this only affect a subset of the supported languages, but you should take the list with a grain of salt.

SemanticDiff includes support for 11+ languages, covering most major languages such as TypeScript/JavaScript, Python, Go, Rust, C# and Java. Adding new languages to SemanticDiff is more involved than in difftastic since additional annotations and rules are needed to perform a semantic comparison. This also places higher demands on the quality of the parsers that are used. That said, neither tool can guarantee that code will always be parsed correctly.

Miscellaneous

So far we have focused on the major differences between difftastic and SemanticDiff, but there are also many smaller ones that may or may not be relevant for you. Here is a small selection:

Indention Changes in Python Code

Difftastic currently does not support showing indention changes in Python code (see #587). This makes it impossible to detect certain changes like the one shown in the diff below. You may want to keep this in mind when diffing Python code or wait for a fix. SemanticDiff does not have such a limitation and will mark the corresponding whitespace characters as changed.

@@ -3,7 +3,7 @@
 def authenticate_user(user, pw_hash):
     if compare_digest(user.password, pw_hash):
         audit.log(f"User {user.username} authenticated")
-        return True
+    return True
 
     audit.log(f"Authentication for {user.username} failed")
     return False

Syntax Highlighting

Difftastic offers a very subtle syntax highlighting compared to most editors and code review platforms. It uses only few colors and the font weight to highlight parts of the code. You can disable it, but there are no further options for customization.

SemanticDiff aims for a seamless integration. The VS Code extension uses the same syntax highlighting and color theme as configured in the VS Code settings. This provides you with a lot of customization options and makes the transition between SemanticDiff and other VS code views almost unnoticeable.

Inline vs Side-By-Side Diff

Both difftastic and SemanticDiff use a Side-By-Side Diff as default. This type of comparison provides a good visualization on how the old and new code are related, which is not trivial if there is no 1-on-1 relationship between lines. Difftastic additionally offers an inline diff mode (also called “unified diff”), that displays the old and new code interleaved.

Summary

Overall, difftastic and SemanticDiff can both help you understand code changes faster. By taking the structure of the code into account they can provide a better matching between the old and new code than a classical diff could. The way they work internally and the features they offer, differ in various areas. We have summarized the most important differences in the table below.

Looking at the table, you can see that difftastic offers support for more languages and is mainly aimed at users who prefer to view their diffs using a terminal or a simple text editor. SemanticDiff focuses on integration with existing tools (VS Code, GitHub) and provides more language specific/semantic features. The diffs generated by SemanticDiff tend to be smaller due to the semantic comparison filtering out irrelevant changes while difftastic highlights all changed tokens.

We hope this comparison has given you some insight into the differences between the two tools and helped you decide which one is right for you. If you are still unsure, you can try both and see which one fits your workflow better and provides the most value.