Standard, the HTML editor detects CSS classes whose names have the prefix .ms-rteCustom. You can easily extend the OOTB styles by adding .ms-rteCustom-XXX to your custom stylesheet. The name "XXX" will then become available in the styles dropdown within the HTML editor.
Another option is to specify a unique CSS class name prefix. This give you the ability to present different custom styles for different HTML editors of the publishing page. To make a unique CSS class, you have to assign a PrefixStyleSheet property to the HTML field control.
Example:
<.PublishingWebControls:RichHtmlField id="Content" FieldName="PublishingPageContent" runat="server" PrefixStyleSheet="myUniqueClass"./>Afterwards you can define the "myUniqueClassCustom" class in you custom stylesheet.
Warning: you have to append the word "Custom" to the class, otherwise it won't work! In my opinion, MS doesn't explain this very well...
Example:
Syntax = .[PrefixStyleSheetPropertyName]Custom-[TitleAvailableInEditor]
.myUniqueClassCustom-Header1
{
font-weight: bold;
font-size: 16px;
}
Read How to: Customize Styles for more information.