The Atacama Cosmology Telescope Polarimeter (ACTPol) is a polarization sensitive upgrade to the Atacama Cosmology Telescope, located at an elevation of 5190 m on Cerro Toco in Chile. ACTPol uses transition edge sensor bolometers coupled to orthomode transducers to measure both the temperature and polarization of the Cosmic Microwave Background (CMB). Calibration of the detector angles is a critical step in producing polarization maps of the CMB. Polarization angle offsets in the detector calibration can cause leakage in polarization from E to B modes and induce a spurious signal in the EB and TB cross correlations, which eliminates our ability to measure potential cosmological sources of EB and TB signals, such as cosmic birefringence. We calibrate the ACTPol detector angles by ray tracing the designed detector angle through the entire optical chain to determine the projection of each detector angle on the sky. The distribution of calibrated detector polarization angles are consistent with a global offset angle from zero when compared to the EB-nulling offset angle, the angle required to null the EB cross-correlation power spectrum. We present the optical modeling process. The detector angles can be cross checked through observations of known polarized sources, whether this be a galactic source or a laboratory reference standard. To cross check the ACTPol detector angles, we use a thin film polarization grid placed in front of the receiver of the telescope, between the receiver and the secondary reflector. Making use of a rapidly rotating half-wave plate (HWP) mount we spin the polarizing grid at a constant speed, polarizing and rotating the incoming atmospheric signal. The resulting sinusoidal signal is used to determine the detector angles. The optical modeling calibration was shown to be consistent with a global offset angle of zero when compared to EB nulling in the first ACTPol results and will continue to be a part of our calibration implementation. The first array of detectors for Advanced ACTPol, the next generation upgrade to ACTPol, will be deployed in 2016. We plan to continue using both techniques and compare them to astrophysical source measurements for the Advanced ACTPol polarization calibration.