From everything that i have read Vitex is safe to take when ttc but it is recommended that you stop taking it when you are pregnant and/or nursing. Some studies show that Vitex in early pregnancy can decrease miscarriage but for the most part it has been inconclusive. But there are some people that take it till the end of the first trimester.
i took vitex when ttc dd and stopped taking it as soon as i got a bfp.
Here are some links:
http://www.ovarian-cysts-pcos.com/vitex.html
http://healthyimmunity.com/Products/Vitex-Chaste-Tree-Berry.html
http://en.wikipedia.org/wiki/Vitex_agnus-castus
And here is some info i copied and pasted about Vitex
Vitex agnus-castus (chaste tree berry) is a well-known
herb that has been used for centuries in Europe for hormonal
imbalances in women.
It may be helpful for women with PCOS who do not have a normal
menstrual
cycle and thus don't ovulate or menstruate. A large
percentage of these menstrual problems are related to
insufficient progesterone during the luteal phase of the
menstrual cycle, which is called a luteal
phase defect or corpus luteum insufficiency. A corpus luteum
insufficiency is defined as an abnormally low progesterone level
3 weeks after the onset of menstruation. Insufficient levels of
progesterone may result in the formation of ovarian cysts.In addition, some PCOS women have too much prolactin, which can
inhibit fertility. Vitex may help with both of these problems
When using chaste tree berry, don’t expect immediate
results. It may not have much effect until it is taken for at
least 3-6 months.
It’s not the same as taking progesterone. You don’t
give it, stop it, and then expect a withdrawal bleed a few days
later as you would if you were taking progesterone or synthetic
progestins.