dss defines secondary structure based on backbone geometry and hydrogen bonding patterns.
With PyMOL, heavy emphasis is placed on cartoon aesthetics, and so both hydrogen bonding patterns and backbone geometry are used in the assignment process. Depending upon the local context, helix and strand assignments are made based on geometry, hydrogen bonding, or both.
This command will generate results which differ slightly from DSSP and other programs. Most deviations occur in borderline or transition regions. Generally speaking, PyMOL is more strict, thus assigning fewer helix/sheet residues, except for partially distorted helices, which PyMOL tends to tolerate.
WARNING: This algorithm has not yet been rigorously validated.
dss selection, state
state = state-index or 0 for all states
If you dislike one or more of the assignments made by dss, you can use the alter command to make changes (followed by "rebuild"). For example:
alter 123-125/, ss='L' alter pk1, ss='S' alter 90/, ss='H' rebuild
Secondary Structure Determination
As is typical with PyMOL, the secondary structure assignment engine is ad hoc and empirically tuned to produce desirable aesthetics. Though there are some phi/psi's that are clearly helix/sheet and others that are clearly not, there are certain regions of phi/psi space were the assignment is subjective or arbitrary. In my experience, algorithms based on strict definitions tend to blow it in such regions, and so PyMOL's algorithm is "fuzzy" in that there is a grey area where residues may be accepted or rejected as helix/sheet depending upon the surrounding context.
There aren't any hard & fast definitions. But you are welcome to check out the collection of settings beginning with "ss_helix" and "ss_strand", noting that the include and exclude settings are devations around the target in degrees. If you think PyMOL is botching an assignment, then you might try varying these.