I've found that the easiest thing for me right now is to use good old paper and pencil! I have a spiral notebook that has become my financial diary. I have a lot of figures in it - total debt, income, lists of bills that are due from paycheck to paycheck and so forth. I don't keep a traditional check register anymore. Since Dave teaches us to use a "Zero-based Budget" I don't see a need to keep a running check register. Instead, before each of Aaron's paychecks, I start a new sheet in my spiral notebook for that period's checking account transactions. The first line is our starting balance which is always $0 because every penny is sent to the debt snowball after bills are paid. The next line, I add Aaron's paycheck amount and the next is our cash out line. (We pull cash for everything we can to limit debit card transactions.) I then start listing the bills that are due (by date). After that, whatever is left goes to the debt snowball. I typically leave about $4-$5 for the first week just in case I've made a math error. Once I'm certain that I haven't - typically by the next week - I transfer those few dollars over to the credit card.
So far this system works for us. I do have a couple of iPad apps that I use. One is a debt snowball app. I like to play with it to figure out our debt free date. You can see a payment schedule as well as interest paid and "fun" stuff like that. (I think it's fun but I am a big nerd.) The other app I use is a simple calendar called Bills. It just keeps track of my monthly bills so I don't miss any. I like to see them placed on a monthly calendar. I also like that I can check them off as I pay them. If I don't check one off and the due date arrives, I get a notification on my iPad. You could do the same type of thing on a printed calendar but the app is free and I just kinda like it. :)
I've had several people suggest Mint as a good budgeting app. I may check it out, but for now I've just found that the old fashioned paper and pencil are working well for us. Maybe as we sharpen our budgeting skills we will dip into some fancy software or apps but for now we are content with our system. :)
Here's a screenshot of the three budgeting apps I'm currently using. I believe the one on the far left was one that I paid around a dollar for because the lite (free) version was limited to just one or two accounts.