Determining the likelihood that a pattern you observed is due to a particular mechanism

Introduction and definitions

The following steps describe a method for comparing and trying to match your observation to model of pattern generation. In this description:

"Observation" means the data, image, or drawing of some phenomenon or list of events in a time series that you collected

"Representation" means how you recorded that observation. This could be a table of discrete data points, a line graph, a digital or binary image, a set of steps, etc. There may be multiple representations that you could use that are equivalent, but not all representations will faithfully represent all forms of data.

A "Mechanism" is a set of interactions or forces that generates patterns. A "Model" is a hypothetical mechanism or model that generates a pattern. In this discussion, these are treated as hypothetical models that are being tested against the data/observation.

"Likelihood" is the ranking or relative belief that a particular mechanism/model is the source of the pattern you observed. It is not the confidence that you have in one model's ability to explain your observed data.

 

This set of steps has several choice divergences and/or loops. You do not proceed through all seven.

1. Make an observation and fix it into one representation.

Some common choices might be to sketch onto a grid, take a picture and transform that digital image onto a coarser grid, take data points (x,y,z) and plot that onto a chart, or create a time-line of events.

You might have to revisit this step to re-represent your observation in another format (step 7).

2. Scan your mental repertoire of patterns for similar patterns.

These patterns will suggest the hypothetical models that could have generated what you observed.

Rank these patterns in terms of how likely they are to explain your observation. This can be an informal or casual list or an attempt to assign relative likelihoods (such as pattern-model A is 3 times more likely than pattern-model B).

 

3. Based on these questions, follow one of the following 4 paths:

a. Is an description of possible models (i.e. without quantification of the likelihood as a probability) sufficient for your use? For example, you might be trying to scope out the possible patterns for further study?

If so, go to 4. Describing the relative likelihood of different hypothetical models in generating something similar your observation.

b. Do you understand the mechanisms for the patterns you have chosen? Do you have a good feeling for how the parameters can change the shape of the pattern? Do you know how to format the models so that they can be compared against the representation of your observation?

If so, go to 5. Challenging your data with hypothetical models.

c. Do you feel that you need to look for more patterns that might be similar to your observation? Do you only have one match and you can't quite think of other patterns or fragments of patterns that might overlap?

if so, go to 6. Browsing and searching the pattern catalog.

d. Do you need to know how to formulate a model and parameterize it so that you can compare it to the representation of your observation? Do you need to know how to set up the equation or create a grid output for the simulation?

if so, go to 7. Selecting and modifying candidate models.

 

4. Describing the relative likelihood of different hypothetical models in generating something similar your observation.

<!-- add in vocabulary to describe this with examples -->

qualitative assessment

descriptive words

5. Challenging your data with hypothetical models.

Take your observed data and challenge it with various models to determine their relative likelihoods. You may be starting with no prior estimates of the probability of the models or you may use the prior probabilities that you assigned in step 2.

6. Browsing and searching the pattern catalog.

Browse or search through the Pattern Catalog* taking time to look at the families of patterns generated by different parameterizations of each model. Also pay attention to the types and units of the parameters (time, distance, etc) because these are salient or highly descriptive features of some hypothetical mechanisms. For example if you are dealing with dimensionless quantities it could be because these are ratios of other quantities and would suggest growth or decay type functions. There are other examples of dimensionless parameters describing an equation.

*The Pattern Catalog is Appendix 3. in "Multiple Perspectives"

 

7. Selecting and modifying candidate models.

Select particular candidate models from the Pattern Catalog and play with the simulations. Pay particular attention to whether you might have to revisit the format you chose for your initial representation of your observation. You should not consider modifying pieces of the observation to fit the hypothesis, that is discouraged in this method.

If you are really trying to quantify the relative likelihood of models in describing your observation, you will have to proceed by referring to other references such as Hilborn and Mangel 1997.

Summary

This is a short version of this process.

• This approach compares mulitple models or simulations to see which has the best fit with your observations. It is not a matter of rejecting a single hypothesis or working through branches of hypotheses, but requires alternative models to be compared.
• Record your observations into your notebook. These data or descriptions can be represented in different ways (text, numbers, graphics, etc). If you recorded data points, this step includes graphing and studying those relationships.
• Select possible models for mechanisms that could generate patterns or relationships similar to what you observed.
• At the first level, estimate which of these models is more likely (strictly, for which model is the data more likely). These should be represented in relative likelihoods, such as model A is 4 times more likely than model B to match the data.
• At the next level, which you may not wish to reach, use your previous estimates as prior probabilities and use inverse statistical techniques to refine the likelihood estimates.

References

Hilborn, R., and Mangel, Marc (1997). The Ecological Detective: Confronting Models with Data. Princeton, NJ, Princeton University Press.

Objects/Models and Hypotheses

teaching/tlc/likelihood-of-pattern.html