I have created a custom login module for my web application running in Wildfly 8.0. Here is the module:
package bmacs.auth;
import java.security.acl.Group;
import javax.security.auth.login.LoginException;
import org.jboss.security.SimpleGroup;
import org.jboss.security.SimplePrincipal;
import org.jboss.security.auth.spi.UsernamePasswordLoginModule;
public class BasicLoginModule extends UsernamePasswordLoginModule
{
    @Override
    protected String getUsersPassword() throws LoginException
    {
        System.out.println("custom getUsersPassword");
        System.out.format("MyLoginModule: authenticating user '%s'\n",
                getUsername());
        String password = super.getUsername();
        password = password.toUpperCase();
        return password;
    }
    @Override
    protected boolean validatePassword(String inputPassword,
            String expectedPassword)
    {
        System.out.println("custom validatePassword");
        String encryptedInputPassword = (inputPassword == null) ? null
                : inputPassword.toUpperCase();
        System.out.format(
                "Validating that (encrypted) input psw '%s' equals to (encrypted) '%s'\n"
                , encryptedInputPassword, expectedPassword);
        return true;
    }
    @Override
    protected Group[] getRoleSets() throws LoginException
    {
        System.out.println("custom getRoleSets");
        SimpleGroup group = new SimpleGroup("Roles");
        try {
            System.out.println("Search here group for user: "+super.getUsername());
            group.addMember(new SimplePrincipal("RoleReportEnrollmentViewer"));
        } catch (Exception e) {
            throw new LoginException("Failed to create group member for " + group);
        }
        return new Group[] { group };
    }
}
Here is my new security domain I added to standalone.xml
<security-domain name="simple-auth" cache-type="default">
                    <authentication>
                        <login-module code="bmacs.auth.BasicLoginModule" flag="required" module="login"/>
                    </authentication>
                </security-domain>
Here is my web app's jboss-web.xml, which references the security domain.
<?xml version="1.0" encoding="UTF-8"?>
<jboss-web>
    <security-domain flushOnSessionInvalidation="true">simple-auth</security-domain>
</jboss-web>
When I try to login (through form authentication J_SECURITY_CHECK), it does nothing. The only thing that shows up in the log is these 2 lines, which isn't much help
16:55:09,622 TRACE [org.jboss.security] (default task-9) PBOX000200: Begin isValid, principal: org.wildfly.extension.undertow.security.AccountImpl$AccountPrincipal@c9c352bc, cache entry: null
16:55:09,625 TRACE [org.jboss.security] (default task-9) PBOX000354: Setting security roles ThreadLocal: null
What am I missing? The System.out.println statements in the custom login never print anything/not being executed.
                        
Check again the jboss-web.xml. I think that you have to setting the security roles in the xml. I give you a url that may help
https://stackoverflow.com/questions/29726261/form-based-authentication-in-wildfly-with-jsf
Tell me something if it works for you.
Regards.