I tried to add Disqus comments to my site yet it didn’t work. It turned out to be a stupid and old mistake. I didn’t activate my newly registered Disqus account by verifying email address.

This nice post helped me to find that out. http://hal.case.edu/~rrc/blog/2013/09/29/disqus-comments-not-visible/

A simple reminder to myself, always not make assumptions about simple little things. We as human being tend to make quick assumptions about things that look simple and trivial to us, thus not giving it a second thought before we hurry to the next step/thing. Compputer Science/Software Development is unfortunately a field in which such careless assumptions could slow you down dramatically or even fatal ( in terms of system not life).

So slow down and make sure everything is tidy and right.

For numerous times I set goals, stumble and fall through the process, due to various factors such as: not seeing progress as expected after making ‘a lot’ of effort, distractions, ‘I am not good enough’. Yet the good thing is that I struggled but never wanted to give up this struggling process.

As I am flowing through life of events, ideas that are pieces scattered around in mind start to fit together. Yet I was still wrestling with myself. Like in the Bible says, things to want to do, you don’t do; things you don’t want to do, you do.

The fundamental question is: what is preventing me from making progress steadily towards my goals? Why self discipline could rust and become weaker?

I tried to answer these questions by reading works of Yangming WANG, an influential Chinese philosopher. His theory is like this: “Everyone of us in this world knows what is right and what is wrong according to their own standards. Ideally, we follow our heart when making any decisions, thus always doing the right thing, thus we could always keep a peaceful mind. However the problem is, we all have all kinds of desires, often against what our standards of right/wrong. This is the source of all struggles.” Based on this, he thinks the single biggest, abstract goal of life is to: “Always following your heart to the extreme, thus exploring the full potential of our capabilities.” He also pointed out clearly, all of this takes practice. Everything we encounter is an opportunity to practice this methodology.

I agree with what he advocates, and it does really work when I try to practice according to his theory. For example, sometimes it is irrestible to react in an angry fashion when my wife and I are disputing upon something. Many times when the words are the tip of my lips, a voice inside told me, “No, no, no, this will not help the situation and possibly make things worse. If what you want is to cool things down and reach agreement, saying so will shooting yourself in foot effectively”, by practicing this, I was able to cut that urge to fight back at the root, which was so intuitive and natural before. Another example is, I used to be reluctant to wash the dishes in the sink until later. Being lazy was the only reason. I decided to practice on this so that everytime I felt to lay back in the chairs to take a break after a meal, a voice told me, “If you are not doing this right now, you are gonna have to wash them later. Even worse, the idea that “there are dishes to wash later” is going to linger behind your mind as a background process, consuming the precious CPU cycles. Moreover, stains will be harder to wash off after drying out, taking more time and water. This is not time & resource efficient.“ This rationale actually would push me to wash those dishes right away, which was not thing I like the most at that moment but a thing to do making a lot of sense.

Gradually, because I am doing more and more things that I should do, and not doing things that I should not do. I started gaining better sense of fulfilment and happiness. Yet there is still a long way to go, I still feel frustrated when my effort seems not generating any results, and I still worry and fear in front of challenges. Thanks to Dr. Carol Dweck’s great work, I finally learned that it is important to BELIEVE the importance to nurture a “growth mindset”, to BELIEVE challenges, errors and trials are great opportunities to learn, to grow my ability and skills, and to know that self-limitting believes such as “I am not good enough so that I am not gonna make it” are unreasonable execuses for not taking actions toward goals.

A Great Book

I was taught many “wisdoms” as a kid. Yet it was not until going through struggles that I internlize those concepts and wisdoms. This is what WANG Yangming said, “You don’t understand something if the word did not become your flesh; you understand the wisdom only when you live it”

Thank you very much WANG Yangming, and Dr. Dweck. You taught me what is a rational outlook towards failures, errors, difficulities, and all I need to do is always apply this in my everyday life.

Now I like my Nike T-shirt better, which says: “JUST DO IT” :)

Here is the problem: you have a WebDAV server, it works with almost all WebDAV clients except Windows 7 mini-redirector when using Digest Authentication.

Admittedly, why choosing Digest Authentication and sticking with Windows 7 mini-redirector in itself might be a debate. This article does not discuss about design options such as this. It only aims to share what I’ve learned struggling with the Microsoft’s WebDAV client so that other folks won’t pay the price in the future.

The usual way to connect to a WebDAV server from Win7 is to open up a Windows Explorer window, map a net drive to the url of the server. If the server is protected by Digest Authentication, you will be prompted to enter your username and password. You type in, submit, and it pops up another box, asking you for credentials again. You keep typing the correct credentials 3 times, and Windows will not allow you keep trying.

This is the problem I was facing. Making things more interesting, the problem could be masked when Fiddler the web debugger is present. That is to say, whenever Fiddler is the man in the middle, it works; otherwise, it stops working.

I tried to approach this problem from many directions which I will cover later in this post, but all didn’t solve the problem.

I was a big step forward when I discovered that Fiddler has two connection related options: “Reuse client connection” and “Reuse server connection”, both are turned on by default for peformance reason I suppose. The working/not working scenarios I described earlier could be reproduced by toggle “Reuse client connection” on/off, without shutting down the Fiddler completely.

By comparing the connection patterns of my session with the session between Win 7 client and Apache, the difference turned out to be that my WebDAV server always drops the connection, especially upon returning 400 series of HTTP status code, for example, 401 Unauthorized. The fix is simple, keeping the connection alive upon 401 solves the problem immediately.

My coworker, a seasoned developer, told me this is an ancient bug of Microsoft, which existed for over 12 years but they never fixed it. The client starts a TCP connection, C, and then sends a plain HTTP request, the server will generate a 401 response together with “WWW-Authenicate” header, including the Digest information, sends it back to the client. At this particular moment, the server has the choices to either keep the connection alive, or drops it, regardless of what the “Connection”, “Keep-Alive” header says earlier. Say the server decided to drop the connection, when the 401 response get to the win 7 client, it will compute an “Authorization” header needed for Digest Authentication, however, win 7 client insists to send this header along through the connection C created earlier, if C is broken, it will start a new connection, C’, send a plain request WITHOUT the “Authorization” header. At this point, you should be able to predict what is going to happen next and to explain why the multiple login problems exist ever.

To summary the above process, the Win 7 client will ONLY send the “Authorization” header upon two conditions: 1. right after you submit the credentials, i.e. when “Authorization” header was created the first time; 2. the connection was the same connection through which it sent the original plain request and got the 401 response.

HTTP is a stateless protocol, neither client nor server should rely on any states such as the connection status. A robust server such as Apache with WebDAV module enabled or a robust client such as Cadaver is able to cope with a rigid client such as win 7 client, or a rigid server such as my server.

WebDAV with Digest is hard to get right, I only saw two servers made it right so far, one is the popular Apache DAV module, the other is my server after fixing this bug.

Win 7 WebDAV support is indeed crappy. There are many other choices for your customers. Cadaver is an excellent open source WebDAV client on Linux/Unix platforms, Mac has build-in WebDAV support, and third party clients such as Cyber Duck, BitKinex, etc. are all good choices. However, if a large portion of your customers are still relying on Windows platform thus Win7 mini-redirector is still their most convenient way to access their WebDAV server, you may still need to make it work for the customers. Here are some other possible causes that the Digest Authentication that does not work.

1. Your authentication logic is implemented wrong so it won’t accept even correct credentials
2. The DAV response body uses default namespace, refer to the links below for further details: http://www.greenbytes.de/tech/webdav/webdav-redirector-list.html#issue-namespace-handling https://issues.apache.org/bugzilla/show_bug.cgi?id=49428
3. If you are sending “Authentication-Info” header, be sure to make it work

If all of these does not help you, here are some approaches I found useful when hunting for the root cause: 1. Use Fiddler, ngrep to capture and study the traffic 2. Use open source clients and servers as base reference. You could know the machinery of process by reading the code; the code is well tested and reliable 3. Expand your perspectives. If HTTP communication does not work, the reason might be the traffic (content), timeout (timing), connection (context),etc.
4. Remember the old fact: HTTP is stateless. No assumptions should be made based on any states added 5. Read the RFC carefully and do not hesitate to ask questions online

To wrap up, Digest Authentication is a scheme stronger than Basic. Basic literally provides no protection in terms of today’s security technologies and Digest is inherently vulnerable to man in the middle attack. Please think carefully which security context are you using them in.