Note
Go to the end to download the full example code.
Matrix Perturbation Correlation Comparison
This example demonstrates how to generate a matrix comparison plot for the perturbation correlation method (described in Matrix of Perturbation Correlation Plots and Perturbation Correlation Plots). This example compares the calculated correlation correlation coefficient to the TSUNAMI-IP calculated \(c_k\) value for a set of dummy SDFs taken becuase they only use cross sections available in the dummy 56 group library. Note before running this example, please ensure that the necessary cross section libraries are cached by running the Perturbation Correlation Plots example.)
Getting TSUNAMI-IP \(c_k\) Values
First, we need to get the TSUNAMI-IP \(c_k\) values for the experiment and application SDFs we want to compare. We can do this
by using the tsunami_ip_utils.integral_indices.get_integral_indices() function, as shown in
Reading Integral Indices.
from tsunami_ip_utils.integral_indices import get_integral_indices
from paths import EXAMPLES
application_sdfs = [ EXAMPLES / 'data' / 'example_sdfs' / 'u235-dummy' / f'sphere_model_{i}.sdf' for i in range(1, 4) ]
experiment_sdfs = application_sdfs
# Get the TSUNAMI-IP integral indices
coverx_library = '56groupcov7.1'
integral_indices = get_integral_indices(application_sdfs, experiment_sdfs, coverx_library=coverx_library)
c_k = integral_indices['c_k']
Generating the Comparison
A matrix comparison plot comparing the computed correlation coefficient to the TSUNAMI-IP \(c_k\) values can be generated using
the tsunami_ip_utils.comparisons.correlation_comparison() function. Note this function is used to compare any method for computing
any of the available TSUNAMI-IP integral indices (\(E\) and \(c_k\)).
from pathlib import Path
from tsunami_ip_utils.comparisons import correlation_comparison
# Define paths used for generating cross section perturbations
multigroup_library = EXAMPLES / 'data' / 'dummy_56_v7.1'
perturbation_factors = Path("~/codes/SCALE-6.3.1/data/perturb/56n.v7.1")
50 Points
First, we’ll generate a comparison for 50 points
num_perturbations = 50
comparisons, fig = correlation_comparison(
integral_index_matrix=c_k,
integral_index_name='c_k',
application_files=application_sdfs,
experiment_files=experiment_sdfs,
method='perturbation',
base_library=multigroup_library,
perturbation_factors=perturbation_factors,
num_perturbations=num_perturbations,
)
fig.show()
print(comparisons)
Generating perturbation points: 0%| | 0/50 [00:00<?, ?it/s]
Generating perturbation points: 66%|██████▌ | 33/50 [00:00<00:00, 328.53it/s]
Generating perturbation points: 100%|██████████| 50/50 [00:00<00:00, 334.99it/s]
Generating perturbation points: 0%| | 0/50 [00:00<?, ?it/s]
Generating perturbation points: 100%|██████████| 50/50 [00:00<00:00, 1581.31it/s]
Generating perturbation points: 0%| | 0/50 [00:00<?, ?it/s]
Generating perturbation points: 44%|████▍ | 22/50 [00:00<00:00, 191.56it/s]
Generating perturbation points: 100%|██████████| 50/50 [00:00<00:00, 378.56it/s]
Generating perturbation points: 0%| | 0/50 [00:00<?, ?it/s]
Generating perturbation points: 100%|██████████| 50/50 [00:00<00:00, 1470.43it/s]
Generating perturbation points: 0%| | 0/50 [00:00<?, ?it/s]
Generating perturbation points: 100%|██████████| 50/50 [00:00<00:00, 1452.48it/s]
Generating perturbation points: 0%| | 0/50 [00:00<?, ?it/s]
Generating perturbation points: 44%|████▍ | 22/50 [00:00<00:00, 163.63it/s]
Generating perturbation points: 100%|██████████| 50/50 [00:00<00:00, 329.29it/s]
Application Number 1 ... 3
Attribute Calculated TSUNAMI-IP Percent Difference ... Calculated TSUNAMI-IP Percent Difference
Experiment Number ...
1 1.000E+00 1.000E+00+/-5.00E-03 0.00+/-5.00E-01 ... 9.954E-01 9.985E-01+/-5.00E-03 0.31+/-4.99E-01
2 9.974E-01 9.994E-01+/-5.00E-03 0.20+/-4.99E-01 ... 9.859E-01 9.998E-01+/-5.00E-03 1.39+/-4.93E-01
3 9.954E-01 9.985E-01+/-5.00E-03 0.31+/-4.99E-01 ... 1.000E+00 1.000E+00+/-5.00E-03 0.00+/-5.00E-01
[3 rows x 9 columns]
From this we can see an interactive matrix plot showing the perturbation correlation plots with the computed pearson and spearman
correlation coefficients and the TSUNAMI-IP computed \(c_k\) values. Note that any plots disagreeing more than 5% are highlighted
in red. In addition, we get a multi-index pandas dataframe showing the comparisons (which can be written to excel as in
Comparing Calculated and TSUNAMI-IP E for a Set of SDF Files).
Note that all of the plot objects are generated using the defaults, and the matrix plot is also generated using the defaults.
If you want to customize the plot, just pass in plot_object_kwargs or matrix_plot_kwargs to the function.
200 Points
Now, using a slightly larger number of points: 200 (remember 1000 is the maximum), we can generate another comparison, but this time using static matplotlib plots and including labels
num_perturbations = 200
labels = {
'applications': [ application_file.name for application_file in application_sdfs ],
'experiments': [ experiment_file.name for experiment_file in experiment_sdfs ],
}
comparisons, fig = correlation_comparison(
integral_index_matrix=c_k,
integral_index_name='c_k',
application_files=application_sdfs,
experiment_files=experiment_sdfs,
method='perturbation',
base_library=multigroup_library,
perturbation_factors=perturbation_factors,
num_perturbations=num_perturbations,
plot_objects_kwargs={'plot_type': 'scatter'},
matrix_plot_kwargs={'labels': labels}
)
fig.show()
print(comparisons)
Generating perturbation points: 0%| | 0/200 [00:00<?, ?it/s]
Generating perturbation points: 56%|█████▌ | 111/200 [00:00<00:00, 1099.98it/s]
Generating perturbation points: 100%|██████████| 200/200 [00:00<00:00, 496.19it/s]
Generating perturbation points: 0%| | 0/200 [00:00<?, ?it/s]
Generating perturbation points: 62%|██████▎ | 125/200 [00:00<00:00, 643.97it/s]
Generating perturbation points: 100%|██████████| 200/200 [00:00<00:00, 826.68it/s]
Generating perturbation points: 0%| | 0/200 [00:00<?, ?it/s]
Generating perturbation points: 72%|███████▏ | 143/200 [00:00<00:00, 620.13it/s]
Generating perturbation points: 100%|██████████| 200/200 [00:00<00:00, 749.66it/s]
Generating perturbation points: 0%| | 0/200 [00:00<?, ?it/s]
Generating perturbation points: 74%|███████▍ | 149/200 [00:00<00:00, 1488.94it/s]
Generating perturbation points: 100%|██████████| 200/200 [00:00<00:00, 1469.33it/s]
Generating perturbation points: 0%| | 0/200 [00:00<?, ?it/s]
Generating perturbation points: 74%|███████▍ | 148/200 [00:00<00:00, 1471.39it/s]
Generating perturbation points: 100%|██████████| 200/200 [00:00<00:00, 1471.32it/s]
Generating perturbation points: 0%| | 0/200 [00:00<?, ?it/s]
Generating perturbation points: 66%|██████▌ | 131/200 [00:00<00:00, 512.98it/s]
Generating perturbation points: 100%|██████████| 200/200 [00:00<00:00, 667.80it/s]
Application Number 1 ... 3
Attribute Calculated TSUNAMI-IP Percent Difference ... Calculated TSUNAMI-IP Percent Difference
Experiment Number ...
1 1.000E+00 1.000E+00+/-5.00E-03 0.00+/-5.00E-01 ... 1.000E+00 9.985E-01+/-5.00E-03 -0.15+/-5.02E-01
2 1.000E+00 9.994E-01+/-5.00E-03 -0.06+/-5.01E-01 ... 1.000E+00 9.998E-01+/-5.00E-03 -0.02+/-5.00E-01
3 1.000E+00 9.985E-01+/-5.00E-03 -0.15+/-5.02E-01 ... 1.000E+00 1.000E+00+/-5.00E-03 0.00+/-5.00E-01
[3 rows x 9 columns]
Note that for these examples, adding more poins hardly changes the agreement, because the TSUNAMI_IP \(c_k\) values are ~1, only a small number of points are needed to get a small uncertainty on the correlation coefficient (see the technical manual for details).
These plots can also be saved to an image
comparisons, fig = correlation_comparison(
integral_index_matrix=c_k,
integral_index_name='c_k',
application_files=application_sdfs,
experiment_files=experiment_sdfs,
method='perturbation',
base_library=multigroup_library,
perturbation_factors=perturbation_factors,
num_perturbations=num_perturbations,
plot_objects_kwargs={'plot_type': 'scatter'},
matrix_plot_kwargs={'labels': labels}
)
fig.to_image( EXAMPLES / '_static' / 'perturbation_matrix_comparison.png' )
# sphinx_gallery_thumbnail_path = '../../examples/_static/perturbation_matrix_comparison.png'
Generating perturbation points: 0%| | 0/200 [00:00<?, ?it/s]
Generating perturbation points: 72%|███████▏ | 143/200 [00:00<00:00, 1420.84it/s]
Generating perturbation points: 100%|██████████| 200/200 [00:00<00:00, 1458.27it/s]
Generating perturbation points: 0%| | 0/200 [00:00<?, ?it/s]
Generating perturbation points: 26%|██▋ | 53/200 [00:00<00:00, 289.04it/s]
Generating perturbation points: 100%|██████████| 200/200 [00:00<00:00, 715.16it/s]
Generating perturbation points: 0%| | 0/200 [00:00<?, ?it/s]
Generating perturbation points: 24%|██▍ | 49/200 [00:00<00:00, 312.14it/s]
Generating perturbation points: 100%|██████████| 200/200 [00:00<00:00, 880.67it/s]
Generating perturbation points: 100%|██████████| 200/200 [00:00<00:00, 776.19it/s]
Generating perturbation points: 0%| | 0/200 [00:00<?, ?it/s]
Generating perturbation points: 44%|████▍ | 88/200 [00:00<00:00, 427.41it/s]
Generating perturbation points: 100%|██████████| 200/200 [00:00<00:00, 719.03it/s]
Generating perturbation points: 0%| | 0/200 [00:00<?, ?it/s]
Generating perturbation points: 75%|███████▌ | 150/200 [00:00<00:00, 1494.52it/s]
Generating perturbation points: 100%|██████████| 200/200 [00:00<00:00, 674.53it/s]
Generating perturbation points: 0%| | 0/200 [00:00<?, ?it/s]
Generating perturbation points: 76%|███████▌ | 151/200 [00:00<00:00, 1502.80it/s]
Generating perturbation points: 100%|██████████| 200/200 [00:00<00:00, 1498.52it/s]
A Comparison Without a Plot
Sometimes generating the plots themselves can be extremely memory intensive when making a matrix plot comparison over an entire
series of critical experiments, so sometimes it is desirable to generate the comparison without the plot. This can be done by
passing make_plot=False to the tsunami_ip_utils.comparisons.correlation_comparison() function. This will return only
the comparisons dataframe.
comparisons = correlation_comparison(
integral_index_matrix=c_k,
integral_index_name='c_k',
application_files=application_sdfs,
experiment_files=experiment_sdfs,
method='perturbation',
base_library=multigroup_library,
perturbation_factors=perturbation_factors,
num_perturbations=num_perturbations,
make_plot=False,
)
print(comparisons)
Application Number 1 ... 3
Attribute Calculated TSUNAMI-IP Percent Difference ... Calculated TSUNAMI-IP Percent Difference
Experiment Number ...
1 1.000E+00 1.000E+00+/-5.00E-03 0.00+/-5.00E-01 ... 9.959E-01 9.985E-01+/-5.00E-03 0.26+/-4.99E-01
2 9.976E-01 9.994E-01+/-5.00E-03 0.18+/-4.99E-01 ... 9.872E-01 9.998E-01+/-5.00E-03 1.26+/-4.94E-01
3 9.959E-01 9.985E-01+/-5.00E-03 0.26+/-4.99E-01 ... 1.000E+00 1.000E+00+/-5.00E-03 0.00+/-5.00E-01
[3 rows x 9 columns]
Note also that, since generating points for perturbation plots can be time consuming, this function by default runs in parallel
with two less than the number of cores available on your machine. This can be changed by passing the num_cores argument.
Total running time of the script: (0 minutes 27.367 seconds)