In order to be able to assess the quality of our brains’ outputs, we much be able to assess the quality of the inputs. Using opaque algorithms to access information interferes with our ability to assess the quality of that information as an input. Opaque algorithms decontextualize information; without that context, we have difficulty developing models of what we know we don’t know. Without that context, we don’t know what information was available to be accessed, and why a particular piece of information was selected over others. Without that context, we are unable to map the blind spots in our knowledge. By filtering the inputs we receive–by choosing information for us–these algorithms shape our thoughts. We must understand the what and how of the shaping to be able to reason about what we do and do not know.
This is profoundly important stuff. Read the whole thing.