A crystal plasticity finite element (CPFE) simulation framework was used to predict the combined detrimental effect of mean stress and defects on the fatigue behavior of aluminum alloy. Experimental data for mean-stress effect on fatigue life and crack growth behavior was obtained on metal inter gas (MIG) welded joints of Al-5083/Al-5.8%Mg alloy plates. A 2D representative model for material’s microstructure was used for the simulations, generated using an anisotropic tessellation algorithm using the EBSD measurements data. A total of 10 different microstructure models were generated for each loading condition. The simulated loadings at different stress ranges and stress ratios (R-ratio) were similar to the experimental conditions for the better comparison of the results. Significant heterogeneity in the distribution of R-ratios and the far-field applied R-ratio was observed. The proposed CPFE simulation framework not only predicted well the effect of defects and mean stress on the fatigue lives, but also the scatter induced in them due to the defects.