You are viewing the evaluation version of WebCATSim. Interested in the full version? Request access here.
Simulate CAT
0/0

Estimated Time Remaining: N/A

Item Pool Configuration

No file selected

Item Pool Information Curve

Simulation True θ Distribution

No file selected

Need a custom theta list? Download sample theta list
N/A

N/A

N/A
Item Selection Algorithm

The item with the highest Fisher Information for the computed ability estimate is administered next.

The item with the closest difficulty (𝛽) parameter to the computed ability estimate is administered next.

A random item is selected from the pool of available items to administered next.


An extension of PFI method that adds a random component whose contribution is more prominent at the beginning of administration and diminishes over administration.

The a-stratified multistage algorithm divides the item pool into K strata based on sorted values of a, where a fixed number of items is administered per stratum starting with the lowest discriminating strata.

A refinement of the a-stratified algorithm where items are divided into blocks based on the b parameter.

A variation of the a-stratified algorithm where items are divided into blocks based on the content codes within the item pool.

CAT Stop Criteria
Analysis
N/A
CAT Metrics
CAT Performance Metrics
This section provides information on overall CAT performance by reporting on the relationships between true theta and estimated theta derived from item administration.
Efficiency
N/A
Bias
N/A
MSE
N/A
RMSE
N/A
Pearson Correlation Coef.
N/A

 
Standard Error Analysis Metrics
This section provides information about the precision of the estimates obtain during CAT and the percent of simulees meeting specific SE thresholds.
Mean SE
N/A
Min SE
N/A
Max SE
N/A
Std. Dev. of SE
N/A
% <= 0.30
N/A
% <= 0.25
N/A
Item Exposure and Utilization
Item Exposures

Summary of Item Exposure and Utilization

N/A

Item Exposure and Utilization Metrics
This section provides information about how many items and with what frequency the items in the pool were used across the total number of simulees (each simulee can also be considered a test form).
Item Pool Size
N/A
Items Exposed
N/A
Over-Exposed (exp. > 0.2)
N/A
Under-Utilized (exp. < 0.02)
N/A
Scaled Chi-Square
N/A
Test-Overlap Rate
N/A
Stop Criteria Analysis
Total Stop Criteria Met

Summary of Stop Criteria

N/A

Stop Criteria Metrics
This section reports the number of simulees meeting each of the stopping rules selected.
Total Simulations
N/A
Fixed Length Test
N/A
Standard Error Criteria
N/A
Number of Items Administered Criteria
N/A
Minimum Item Information Criteria
N/A
∆θ Minimum Threshold
N/A
|θ−𝛽| Minimum Threshold
N/A
% Met Stop Criteria vs Final Ability Estimate
Final Ability Estimate Analysis
Average Final Ability Estimate vs Simulated θ

The figure above is the plot of the average final ability estimate across all simulated θ values. For any given simulated θ, the final ability estimates determined for all simulations at that θ is averaged.

Average Simulated θ vs Final Ability Estimate

The figure above is the plot of the average simulated θ across all determined final ability estimates. For any given final ability estimate, the simulated θ at each reporting of that estimate is averaged.

Average Standard Error vs Final Ability Estimate

The figure above is the plot of the average standard error across all determined final ability estimates. For any given final ability estimate, the standard error at each reporting of that estimate is averaged.

Average Test Information vs Final Ability Estimate

The figure above is the plot of the average test information across all determined final ability estimates. For any given final ability estimate, the test information at each reporting of that estimate is averaged.

Item Administration Analysis
Average Number of Items Administered vs Final Ability Score

The figure above is the plot of the average number of items administered across all determined final ability estimates. For any given final ability estimate, the number of items administered at each reporting of that estimate is averaged.

Item Administration Metrics
Average Number of Items Administered
N/A
Content Balancing Analysis
Content Code vs Target Weight and Mean Administration

The figure above compares the desired target weight of administration for a given content code to the average observed percentage of item administration for the same content code across all simulations.

Ranged Detail Analysis
Ranged Detail: Average Standard Error vs Final Ability Estimate
Detail Adjustments
Final Ability Estimate Detail Origin
θ Range
Ranged Detail Metrics
Mean SE
N/A
% <= 0.30
N/A
% <= 0.25
N/A