Insidehunter642

posted 11 years ago

1. HttpClient provides full support for HTTP over Secure Sockets Layer (SSL) or IETF Transport Layer Security (TLS) protocols by leveraging the Java Secure Socket Extension (JSSE). JSSE has been integrated into the Java 2 platform as of version 1.4 and works with HttpClient out of the box.
2. Context Logging Context logging contains information about the internal operation of HttpClient as it performs HTTP requests. Each class has its own log named according to the class's fully qualified name. For example the class HttpClient has a log named org.apache.commons.httpclient.HttpClient.

• Optional 'thank-you' note:
  

Hi all,
I've got a really big problem. It's about to logging into a https-website with HttpClient.
Here is the site : https://www-930.ibm.com/support/esc/signin.jsp
I've also prepared a testaccount for you to test, whether your ideas work or not.

Apache HttpClient is very widely used for sending HTTP requests from java program itself. If you are using Maven, then you can add below dependencies and it will include all other required dependencies for using Apache HttpClient.

Username:

1337iceskater@arcor.de

PW:

1337ic3skat3r

The task is to log in to the site and then work automatically on this page. But it fails at the login, the working process shouldn't be the greatest problem (I hope so).
So here is my code:
import java.net.URLEncoder; import org.apache.commons.httpclient.*; import org.apache.commons.httpclient.methods.GetMethod; import org.apache.commons.httpclient.methods.PostMethod; import org.apache.commons.httpclient.cookie.*; public class Webrobbi { private HttpClient client; //set all informations Live-Http-Headers from Mozilla shows public void setHeader(HttpMethod m){ m.setRequestHeader('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.3) Gecko/20090824 (CK-IBM) Firefox/3.5.3'); m.setRequestHeader('Accept','text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'); m.setRequestHeader('Accept-Language','de-de,de;q=0.8,en-us;q=0.5,en;q=0.3'); m.setRequestHeader('Accept-Encoding','gzip,deflate'); m.setRequestHeader('Accept-Charset','ISO-8859-1,utf-8;q=0.7,*;q=0.7'); m.setRequestHeader('Keep-Alive','300'); m.setRequestHeader('Connection','keep-alive'); m.setRequestHeader('Referer','https://www-930.ibm.com/support/esc/signin.jsp'); } public void login(){ try{ String host = 'www-930.ibm.com'; String protocol = 'https'; String signIn = '/support/esc/signin.jsp'; String check = '/support/esc/j_security_check'; String home = '/support/esc/home.jsp'; //is for working with it later client = new HttpClient(); client.getHostConfiguration().setHost(host, 443, protocol); //get the logging in page to get cookies GetMethod get = new GetMethod(signIn); this.setHeader(get); get.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY); client.executeMethod(get); get.releaseConnection(); System.out.println('Login form get: ' + get.getStatusLine().toString()); //checks for cookies and get them Cookie[] cookies = client.getState().getCookies(); for (int i = 0; i < cookies.length; i++) { Cookie cookie = cookies[i]; System.out.println( 'Cookie: ' + cookie.getName() + ', Value: ' + cookie.getValue() + ', IsPersistent?: ' + cookie.isPersistent() + ', Expiry Date: ' + cookie.getExpiryDate() + ', Comment: ' + cookie.getComment()); } //the posting method for logging in PostMethod post = new PostMethod(check); post.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY); this.setHeader(post); NameValuePair username = new NameValuePair('j_username', URLEncoder.encode('1337iceskater@arcor.de', 'UTF-8')); NameValuePair pw = new NameValuePair('j_password', URLEncoder.encode('1337ic3skat3r', 'UTF-8')); NameValuePair x = new NameValuePair('x', '40'); NameValuePair y = new NameValuePair('y', '20'); post.setRequestBody(new NameValuePair[]{username, pw, x, y}); client.executeMethod(post); post.releaseConnection(); System.out.println('Login form post: ' + post.getStatusLine().toString()); //check for cookies and get them cookies = client.getState().getCookies(); System.out.println('Logon cookies:'); for (int i = 0; i < cookies.length; i++) { Cookie cookie = cookies[i]; System.out.println( 'Cookie: ' + cookie.getName() + ', Value: ' + cookie.getValue() + ', IsPersistent?: ' + cookie.isPersistent() + ', Expiry Date: ' + cookie.getExpiryDate() + ', Comment: ' + cookie.getComment()); } int statuscode = post.getStatusCode(); //checks whether the page is doing a redirect (thats normal when successful logged in or log in failed) if((statuscode HttpStatus.SC_MOVED_TEMPORARILY) || (statuscode HttpStatus.SC_MOVED_PERMANENTLY) || (statuscode HttpStatus.SC_SEE_OTHER) || (statuscode HttpStatus.SC_TEMPORARY_REDIRECT)) { Header header = post.getResponseHeader('location'); if (header != null) { String newuri = header.getValue(); if ((newuri null) || (newuri.equals('))) { newuri = '/'; } System.out.println('Redirect target: ' + newuri); GetMethod redirect = new GetMethod(newuri); client.executeMethod(redirect); System.out.println('Redirect: ' + redirect.getStatusLine().toString()); // release any connection resources used by the method redirect.releaseConnection(); } else{ System.out.println('Invalid redirect'); System.exit(1); } } } catch(Exception e){ e.printStackTrace(); } } //prints the htmlcode of the site 'uri' public void out(String uri){ try{ GetMethod profile = new GetMethod(uri); this.setHeader(profile); profile.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY); this.client.executeMethod(profile); String siteStream = profile.getResponseBodyAsString(); System.out.println(siteStream); } catch(Exception e){ e.printStackTrace(); } } public static void main(String[] args){ Webrobbi wr = new Webrobbi(); wr.login(); } }
I hope you have an idea and/or a solution for my problem =)
Thanks in advantage
Stefanposted 11 years ago

Apache Httpclient Logging Slf4j

• Optional 'thank-you' note:
  

posted 11 years ago

• Optional 'thank-you' note:
  

Thanks for your fast response!
The other tool would be a possibility. But the problem is shown at this piece of code:
//checks whether the page is doing a redirect (thats normal when successful logged in or log in failed) if((statuscode HttpStatus.SC_MOVED_TEMPORARILY) || (statuscode HttpStatus.SC_MOVED_PERMANENTLY) || (statuscode HttpStatus.SC_SEE_OTHER) || (statuscode HttpStatus.SC_TEMPORARY_REDIRECT)) { Header header = post.getResponseHeader('location'); if (header != null) { String newuri = header.getValue(); if ((newuri null) || (newuri.equals('))) { newuri = '/'; } System.out.println('Redirect target: ' + newuri); GetMethod redirect = new GetMethod(newuri); client.executeMethod(redirect); System.out.println('Redirect: ' + redirect.getStatusLine().toString()); // release any connection resources used by the method redirect.releaseConnection(); this.out(newuri); //I forgott this to add, here is the problem shown } else{ System.out.println('Invalid redirect'); System.exit(1); } }
That means that everything seems to be ok. But if I call the page, to which the program was redirected, it tells me, that it is at the log in page and on the log in page is a comment

Your login has failed. Please try again.

I think it could be because of the cookiehandling, but it's only a guess.
If I would change the tool to one of yours, could you give me a good tutorial or a hint how I should start?
Thanks in advantage
Stefanposted 11 years ago

• Optional 'thank-you' note:
  

posted 11 years ago

• Optional 'thank-you' note:
  

If I would change the tool to one of yours, could you give me a good tutorial or a hint how I should start?

Just to be clear, I'm not affiliated with these projects, except for having been an occasional satisfied user. For HtmlUnit, start with the 'How do I...' pages on http://htmlunit.sourceforge.net/. For jWebUnit, examples can be found at http://jwebunit.sourceforge.net/quickstart.html

my plan is not to test a website.

That's what I gathered. Both tools can be used outside of any test scenario as general-purpose tools for programmatic web access.posted 11 years ago

• Optional 'thank-you' note:
  

Httpclient C# Logging

posted 11 years ago

Apache Httpclient Log Request

• Optional 'thank-you' note:
  

Apache Httpclient Logging Tools