NP Bayes reading group (9/27): hierarchical DPs

Our second NPB reading group meeting took aim at the seminal 2006 paper (with >1000 citations!) by Teh, Jordan, Beal & Blei on Hierarchical Dirichlet Processes. We were joined by newcomers Piyush Rai (newly arrived SSC postdoc), and Ph.D. students Dan Garrette (CS) and Liang Sun (mathematics), both of whom have experience with natural language models.

We established a few basic properties of the hierarchical DP, such as the the fact that it involves creating dependencies between DPs by endowing them with a common base measure, which is itself sampled from a DP. That is:

  • G_0 \sim DP(\gamma, H)     (“global measure” sampled from DP with base measure H and concentration \gamma).
  • G_j|\alpha_0,G_0 \sim DP(\alpha_0,G_0)  (sequence of conditionally independent random measures with common base measure G_0, e.g., G_j are distributions over clusters from data collected on different days)

Beyond this, we got bogged down in confusion over metaphors and interpretations, unclear whether G_j‘s were topics or documents or tables or restaurants or ethnicities, and were hampered by having two different version of the manuscript floating around with different page numbers and figures.
This week: we’ll take up where we left off, focusing on Section 4 (“Hierarchical Dicirhlet Processes”) with discussion led by Piyush.  We’ll agree to show up with the same (“official journal”) version of the manuscript, available: here.

Time: 4:00 PM, Thursday, Oct 4.
Location: SEA 5.106
Please email pillow AT if you’d like to be added to the announcement list.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s