Multinomial Naïve Bayes

I am watching lectures on Natural Language Processing. In the week 3 lectures, professor talks about text classification. I found writing the formulas in words helps.

Given a training set, classify test set into a class

We need to calculate two probabilities
Say, we have a training set of 5 documents , with 3 documents of class ‘A’, and 2 documents of class ‘B’.

1) Probability of a class A, given a training set

= number of documents classified as ‘A’\total number of documents in training set

2) Probability of each word in the vocabulary

  • Vocabulary (V) – unique words in the training set
  • Assume there are 3 words in training set hello, world,goodbye
  • All documents for class ‘A’ are merged and same for class ‘B’
  • Probability of a word ‘hello’ given class ‘A’

    = number of times word ‘hello’ occurs in documents classified as ‘A’ + 1 \total words in documents classified as ‘A’ + V

Then, we tackle the test set. And, figure out which class is proportionally having maximum probability. How ?

1 ) Use the prior probability of a class, say ‘A’
2) And take each word in the test set, use the corresponding probability of the word in that class(‘A’) ^ frequency word in the test set
3) And, just multiply…

Teaching English

Me and Rahul are working as volunteers. We help students in speaking English. We go to Kotak Unnati Center, at Goregaon every Saturday. This saturday we decided to give students words, using which they had to form sentences.

I, We, She, He, They is, were, was, here play, go, brush, read garden, movie, book, teeth

We told students to form sentences using words from at-least 3 columns. For example – ” He is playing in the garden”. Then we divided them into two groups; the winner will be the team with maximum correct sentences. We gave them about 10 minutes to come up with sentences.

Some observations -

1) The students enjoyed the exercise. Each student wrote and spoke at-least one sentence. A team could raise objection if they felt the sentence spoken by the other team was wrong.

2) We helped them to understand about Articles – a, an , the. The common mistake some students were making was, missing “a” or “the”, using “a” like “we were watching a cricket”.

3) The students were told about sentence structure, tenses.

4) We had a discussion with the center head regarding the need for a curriculum.

5) We need to come prepared with some lesson plan.

6) English is a strange language. Some of the confusion we tried to clarify was – when to use see\ watch, shall\will. I did some research on google, and found that shall is rarely used in American English.

Use shall in the first person and will in the second and third persons for the simple future tense:
I shall sing this afternoon.
You will succeed.
He will stay at home.

7) Teaching is a 2-way street. I am both, a teacher and a learner in this situation. I have knowledge which I am sharing. But, I have to learn more, and learn ways to share the knowledge.

“If a child can’t learn the way we teach, maybe we should teach the way they learn.”

“To learn and never be filled, is wisdom; to teach and never be weary, is love.”

History..boring,Think Again

During my school days, BBC World Service(English as well as Hindi channel) was my favorite source for news.One of the great things on BBC was that you always got insight into an important news-story, with in-depth coverage of historical background.So,if they were covering Israel-Palestine conflict,you will get to know about various people involved,their history etc.But, when it came to history taught in school, it was quite bland.Though, I think, my history teacher made the subject quite interesting.

Recently, I have become fascinated with maths.My current interest was sparked by this article 2 years back.

“The first thing to understand is that mathematics is an art. The difference between math and the other arts, such as music and painting, is that our culture does not recognize it as such.”

So, true. I have never thought about maths as an art.

“A mathematician, like a painter or poet, is a maker of patterns. If his patterns are more permanent than theirs, it is because they are made with ideas.”

The main problem with school mathematics is that there are no problems.But a problem, a genuine honest-to-goodness natural human question— that’s another thing.
How long is the diagonal of a cube? Do prime numbers keep going on forever?

And my favorite line

“Mathematics is not a language, it’s an adventure.

It made me realize that maths I learnt at school and the way maths was taught, was so boring. Since then, I have picked up a few maths book which focussed on the history.My first book was Journey Through Genius.The most fascinating character I came across was Euclid.He wrote a book called Elements way back in 300 B.C.This book had 465 propositions from plane and solid geometry and the number theory. And, I realized that most of the geometry I did in school was written in this book!!.You can find the book here. Then, I read a novel, a certain ambiguity. This book touches on various topics related to maths like infinite,euclidean\non-euclidean geometry and philosophy.Brilliant book!!
These days, I am reading, The Equation That Could Not Be solved. A book about symmetry ,theory of equations and the creation of group theory.The 2 main protagonists Abel and Galois, both died young, aged 26 and 20 respectively.Their story is both fascinating and tragic.

So, what I have learned in this journey so far? I have learned about how maths and the mathematical ideas have evolved, the great mathematicians who have influenced the evolution of maths, the idea about patterns(sum), the difficulty of getting recognition for one’s work(maybe not these days), great partnerships (Hardy-Ramanujan).
So, my journey continues in rediscovering maths through characters who influenced maths in such a profound way.

What about you? Have you found any subject fascinating through the lens of history?

Lesson of the week – seek->sense->share

It’s amazing, when you come across something which strongly resonates with you.I came across this idea about learning and sharing, Active sense making.

This week I had this experience with a fellow developer.He has joined recently our team, and had little experience in writing unit tests.I had directed him to various online resources, and some books.But, the best way to learn is to start writing tests.He was struggling to write test for a behavior.So, using my own knowledge and experience, and my sense making – I suggested him to think in this way –

“given” there are 2 orders having “Chocolate” as items,”when” the user searches by typing “Chocolate”, “then” the user should see the 2 orders

And, he immediately grasped as to what approach was needed to write tests.