Sorting Algorithms Studies: Insertion Sort

Continuing the Sorting Algorithms Studies series, let’s talk about Insertion Sort. Basically, this algorithm takes inspiration from the way that you would normally sort a hand of shuffled cards.

Explanation

This algorithm works by, starting at the second element of the list, repeatedly comparing the current value with the previous value, and swapping them if the previous value is higher than the current value; or, if the sorting is being done in decrescent order, if the previous value is lower than the current value. Having provided the clarifications, from now on this article will focus on the crescent order implementation.

Besides doing that with the previous element, if there was a swap, before going ahead and making the comparisons amongst the next elements, it goes retroactively, verifying if the swapped element is in the right place. If it isn’t, continues swapping it with the previous elements until it is. After that, the algorithm resumes the comparisons with the next elements.

Structure

It is comprised by two nested loops:

  • The outer one loops amongst the numbers that comprise the range between 1 and the size of the array, minus 1. Example: if the array has size 5, the outer loop will loop from 1 to 4. Let’s call the number of the outer loop x.
  • The inner loop is the one that actually makes the comparisons and swaps. It loops amongst the numbers that comprise the range between x and 0, not including zero itself. Let’s call the number of the inner loop y. But, although the loop is comprised by these values, an inner loop iteration will only be executed if the element in the y index is lower than the element in the y - 1 index. Example: if the array has size 5, the inner loop will loop from 2 to 0 on the first outer loop, 3 to 0 in the second outer loop, and so on, always checking the condition stated above.
  • For each y in the inner loop, as the condition above was already checked as a requirement for the loop iteration to be valid, all that is left is to swap the element in the index y with the one in the index y - 1.

Also, there is an Insertion Sort implementation with a minor optimization. Instead of continuously swapping elements, for each outer loop, it stores the value of the element in the x index in an auxiliary variable. Then, it keeps assigning the value in the index y to the y + 1 index as long as y is bigger or equal to zero and the value in the index y is greater than the value in the auxiliary variable. Finally, when the inner loop is finished, the auxiliary variable value is assigned to the y + 1 index.

Code

This is the implementation of Insertion Sort in Python. It may look slighter more complex than the usual Insertion Sort implementation, but that’s because I’ll to provide codes that work both for the crescent and decrescent sorting.

And this is the implementation of optimized Insertion Sort in Python. Again, this codes works both for the crescent and decrescent sorting.

Finally, here’s the interactive algorithm visualization, with step-by-step execution.

Complexity

Worst-case performance: O(n²)

Best-case performance: O(n)

Average performance: O(n²)

Space complexity 1: O(n)

Auxiliary space 2: O(2) for the normal implementation; O(2) for the optimized implementation. 3

Benchmark

As the posts from the Sorting Algorithms Studies are posted, I’ll benchmark the algorithms against each other. For now, there’s only Bubble Sort and Insertion Sort to be shown.

Sorting 1000 different arrays with 1000 randomly sorted elements
Algorithm Average Time Spent (seconds)
Crescent Decrescent
Insertion Sort Optimized 0.0753 0.0736
Insertion Sort 0.1372 0.1312
Bubble Sort 0.1582 0.1544
Bubble Sort Optimized 0.1596 0.1566

Sorting 1000 different arrays with 1000 elements in ascending order
Algorithm Average Time Spent (seconds)
Crescent Decrescent
Bubble Sort Optimized 0.0001 0.2260
Insertion Sort 0.0002 0.2700
Insertion Sort Optimized 0.0003 0.1450
Bubble Sort 0.0839 0.2211

Sorting 1000 different arrays with 1000 elements in descending order
Algorithm Average Time Spent (seconds)
Crescent Decrescent
Bubble Sort Optimized 0.2533 0.0002
Insertion Sort 0.2929 0.0002
Insertion Sort Optimized 0.1607 0.0003
Bubble Sort 0.2491 0.0919

Wrapping Up

Insertion Sort is a simple algorithm, one that we use daily more than we realize. As the Bubble Sort, it doesn’t performs well, but it may be enough for some scenarios, like when you have really small lists.

Stay tuned for the next chapters of this series!

  1. Auxiliary Space + Input space

  2. Extra or temporary space used by the algorithm during it's execution.

  3. Python's way of swapping two values dismisses the need of a temp variable to make the swap of values. Because of that, there normal implementation in Python uses O(1) auxiliary space. The optimized implementation still uses O(2) auxiliary space.