RAMP事务

作者: yangzhe1991 分类: 我是搞技术的 发布时间: 2015-08-09 14:11 ė 654条评论

最近刚发现Cassandra计划在未来支持“RAMP事务”。这是SIGMOD2014的论文《Scalable Atomic Visibility with RAMP Transactions》引入的概念。本文算是读论文的笔记吧。

RAMP的全称是“Read Atomic MultiPartition”,就是分布式的“读原子性”。文章定义出一种新的隔离性——Read Atomic Isolation。简单说就是在分布式数据库中一个对多行同时写入的事务,需要保证其他客户端在读取事务发生时要么可以见到其修改的所有行,要么都见不到。相对来说,ACID中的A是描述的是“写原子性”,写入多行的事务要么都成功要么都不成功。而Cassandra已经通过logged batch来实现了写原子性。读原子性只保证了单个写入事务在其他读取事务面前的原子性,但是多个事务同时写的话不保证隔离性。所以RA隔离性的事务可以满足诸如“不能在某个时刻同时读到A的好友里有B,而B的好友里没A的状态”这样的需求,因为设置好友关系只需要两条写入指令。经典的“银行转账”类跨行事务的需求因为涉及到并发写需要隔离出一个snapshot来判断用户是否有余额,也就是写之前还得读并且要保证另一个事务读的时候必须是当前事务更新完结果之后也就是要对读加锁,RA隔离性是搞不定的了。用隔离性的术语来说,RAMP是保证了Read Committed级别,但做不到Repeatable reads。这就意味着RAMP事务只满足了一部分需求,当然性能也会比满足大部分需求的要好。

在介绍算法前,论文对场景也做了一些限制,如不同client的事务不会互相阻塞对方、每个节点挂了都只影响其保存的数据的读写而不应该影响其他节点上数据的读写。文章称其为“scalability criteria”,后面提供的算法在保证scalability criteria的情况下保证了读原子性。

文章提出了实现RAMP的算法。总体来说思路是读事务通过读取对应数据的metadata来判断是否遇到了进行一半的写事务;而写事务使用二阶段提交来保证读事务要么都读要么都不读。文章提供了三种具体的算法,来在需要的metadata的数量和额外的读取次数之间做tradeoff。分别叫RAMP-Fast、RAMP-Small、RAMP-Hybrid。

RAMP-Fast

算法中读事务需要1-2轮通信,写事务需要2轮通信。

服务端需要存每个key的每个版本的value及对应的时间戳(ts)和metadata(md),并且每个key存一个最近一次commit的时间戳。服务端在写入的时候有prepare和commit两个操作,prepare将key、value、ts、md存入,而commit时将对应key的latestCommit更新为max(latestCommit, ts),prepare和commit的时间戳由客户端提供。客户端在写入的时候首先生成一个时间戳,并针对每个key找到所需要存储该key的节点去执行prepare操作,其中md为除了这个key之外该事务所要写的其他所有key的集合。每个节点都执行完prepare后再在每个节点执行commit操作,二阶段提交完成。

服务端在读的时候可以指定一个时间戳来读,也可以设0读取lastestCommit对应的时间戳的值,时间戳和metadata也一并返回。客户端在读的时候写在每个key对应节点上以ts=0来读,拿到的一坨东西挨个解析,每个key会返回该key在该节点最近一次事务提交的时间戳和当时提交的其余n-1个key,二维循环挨个比对每个key在所有读回来的事务里最大的时间戳,如果发现某个key读回来的value对应的时间戳不如在其他key所带回的md中该key的时间戳大,就再去对应节点读指定时间戳的value。因为别的节点commit过必然意味着那条事务的所有key在所有节点prepare过,肯定是能读到对应时间戳的value的,因此某次事务一旦有一个客户端读到了其中一个被commit的key必然可以保证其他key也可以去对应的节点读到,进而整个事务的所有行都是完整读到的,保证了读原子性。

简单粗暴但是需要md里带n-1个key,冗余信息较多,但来回通信的轮数较少,写需要固定两轮,读根据情况需要1轮或两轮。当且仅当某个写入事务没有完全commit完并且要读取的key中有两个key恰好分别在这次写入事务中被commit和还没commit才需要第二轮。

此外,只要保证不同client的时间戳永远不一样,而同client的时间戳也永远不一样(当然要保证单调递增但这个在单机很容易),就一定可以保证对应事务全读出来,哪怕不同节点上的系统时间戳有较大误差也不怕(无非是导致可能后写的反而被先写的覆盖因此读不到,但至少是都读不到)。因此需要在时间戳上加入client id来确保不同节电时间戳永远不会一样。

 

RAMP-Small

因为RAMP-Fast需要的额外空间取决于每次事务所写的key数,如果每次都写很多个key会很耗费空间。因此又有个不需要metadata的算法,但每次读取都需要两轮通信。

服务端的prepare和commit和RAMP-Fast没区别,但是读取操作要传的并非是一个特定的时间戳而是一个时间戳的集合。如果集合为空返回lastedCommit对应的版本的ts(不需要value),而如果是查询一个时间戳集合,把集合中的时间戳从大到小分别查询,一旦查到该key有某个版本的时间戳与之相等就返回这个版本的value和ts。

客户端的写入也和RAMP-Fast没区别只是不需要metadata了。第一轮读取设时间戳集合为空,把每个key的commit ts都读回来,每个key所返回的时间戳作为第二轮查询的时间戳集合。因为第一轮拿到的所有commit ts在第二轮都传入每个节点查询,会保证每个key都尽量返回这个ts集合中所包含的最大时间戳对应的版本的value。所以也不会出现某次事务有些key能读有些key读不到的情况。

读取的轮数多了,但需要的空间和传输数据的量都小了。

RAMP-Hybrid

前两组基本算法的核心思想极其正确性的证明都很容易理解——写入时每行都成功prepare才可能commit,一旦某行被commit必然可以把这次事务的全部其他行都按时间戳找出来。区别只是是否有metadata因而是否需要第二次读取。

而第三种RAMP-Hybrid算法,写入时是在metadata中记录该次事务其他key集合对应的bloom filter而非Fast中完整的key集合。于是很容易想明白,第一轮读取时把lastedCommit的ts和value和md都返回,然后把这次读取事务需要的其他key在这个bloom filter过一遍。因为bloom filter能判断一个元素是一定不在这个集合还是大概率在这个集合,所以如果返回的commit ts比其他某个key拿回来的commit ts小就不管了,如果大就过一遍bloom filter,一旦返回false就省去第二次,而返回true就去第二轮拿对应ts的value,当然可能是没有的因为bloom filter有false positive的可能。

因而Hybrid算法需要的额外空间是一个bloom filter,几乎是个比较小的常量级的空间(而非正比于每次写入事务中key的数量)并且可以根据集合数量和允许的误差度调整。并且和Fast算法中的需要第二轮读取的概率比只是大了bloom filter发生false positive时的概率。如果false positive率为0,Hybrid相当于是Fast;如果为1,会约等于退化成Small。而false positive的概率很小。

 

算法的核心说到这已经结束,很多人可能觉得自己写一个出来都不难。

接下来首先讲了一些实现问题,最重要的是垃圾回收,哪些数据可以回收是显而易见的——比lastedCommit小的版本的数据全可以不要了。但是存在一些旧版本的读可能还需要读小一些的数据因为他可能在读第一轮的时候不知道新的commit,这时候强制reader重新读也可以,更好的方式是旧数据留一段时间再删。然后他也说这个算法中写操作被简化成了类似Cassandra的“highest-timestamp-wins policy”,如果不想这样可以把lastedCommit变成一组版本号然后自己去靠一个特定的合并逻辑来解决之类的,但总体上感觉意义不大毕竟人工合并并不能解决真正复杂的问题。

然后又讲了一些优化,比如如果get指定时间戳的时候服务器发现这个时间戳比当前key的lastedCommit大,说明肯定有另一个事务包含这个key并且已经在别的节点commit只是没在这个节点commit,那么server可以直接把这个key也给commit了。这样其他reader再读的时候遇到不一致的概率会低。还比如一旦所有key都commit了其实metadata已经可以不用了那么就异步的清理掉。再有就是如果不着急让其他人读到,writer可以在prepare完成就返回以提高延迟,commit操作异步完成。这些应该都不难想到。

后面是一些实验结果和证明,没细看。

 

 

总结来说,这个算法解决了一个跨行事务“要么都可见,要么都不可见”的需求,注意这并不能保证这次事务的所有值都能读到,因为可能有一个新的事务覆盖了其中的若干个key,读写尤其是并发的写是完全不加锁的,思想很像cassandra一直遵循的“时间戳谁大谁赢靠NTP减少错误”。相对应的Google的Percolator是对读写进行加锁,需要一个全局递增的时间戳,因而可以处理诸如转账之类的严格要求顺序的事务,当然性能也会差一些。

 

 

 

本文出自 杨肉的演讲台,转载时请注明出处及相应链接。

本文永久链接: https://yangzhe1991.org/blog/2015/08/ramp-transactions/

54条评论

  1. Spealared 2022 年 11 月 19 日 09:46 回复
    Google Chrome 90.0.4430.85 Google Chrome 90.0.4430.85 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36

    However, tools other than the one we finally decided to use did not find additional interactions in preliminary tests with random samples non prescription cialis online pharmacy

  2. Outsify 2022 年 11 月 27 日 10:57 回复
    Microsoft Edge 90.0.818.49 Microsoft Edge 90.0.818.49 Windows 8.1 x64 Edition Windows 8.1 x64 Edition
    Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36 Edg/90.0.818.49

    29 achieved to up to 250 400 nmol l by deleting 11 amino acids in the N terminal region, whereas the native version did not express at all generic cialis no prescription

  3. Hampannef 2023 年 1 月 27 日 08:11 回复
    Microsoft Edge 90.0.818.49 Microsoft Edge 90.0.818.49 Windows 7 x64 Edition Windows 7 x64 Edition
    Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36 Edg/90.0.818.49

    tylenol coupon para micardis hct A winter storm watch is in effect through Friday for the Plains states, meaning forecasters expect heavy snow, strong winds and cold rain that could cause difficult travel conditions prescriptions for propecia in nj Imtiaz S, Siddiqui N, Raza SA, Loya A, Muhammad A

  4. JamesBut 2023 年 1 月 27 日 13:02 回复
    Microsoft Edge 90.0.818.42 Microsoft Edge 90.0.818.42 Windows 8.1 x64 Edition Windows 8.1 x64 Edition
    Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36 Edg/90.0.818.42

    https://over-the-counter-drug.com/# over the counter antibiotic

  5. RonaldObevy 2023 年 1 月 28 日 16:44 回复
    Google Chrome 90.0.4430.72 Google Chrome 90.0.4430.72 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36

    what is minocycline prescribed for ivermectin lotion for scabies
    https://stromectol.science/# what is minocycline
    buy amoxil generic amoxil 500 mg

  6. Jesselemia 2023 年 1 月 29 日 02:20 回复
    Microsoft Edge 90.0.818.42 Microsoft Edge 90.0.818.42 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36 Edg/90.0.818.42

    buy doxycycline 100mg doxycycline 100mg or doxycycline hyclate
    http://holidaygiftdelivery.com/__media__/js/netsoltrademark.php?d=over-the-counter-drug.com online doxycycline
    doxycycline hyclate 100 mg cap buy doxycycline without prescription uk and doxycycline without a prescription buy doxycycline 100mg

  7. Rafaelkek 2023 年 1 月 29 日 03:39 回复
    Google Chrome 90.0.4430.72 Google Chrome 90.0.4430.72 Windows 8.1 Windows 8.1
    Mozilla/5.0 (Windows NT 6.3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36

    https://zithromax.science/# zithromax 500 mg for sale
    cheap amoxil amoxicillin over the counter in canada

  8. RonaldWorce 2023 年 1 月 29 日 04:02 回复
    Microsoft Edge 90.0.818.42 Microsoft Edge 90.0.818.42 Windows 8.1 x64 Edition Windows 8.1 x64 Edition
    Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36 Edg/90.0.818.42

    minocycline generic stromectol
    https://stromectol.science/# ivermectin 0.5%
    doxycycline 50mg doxycycline 200 mg

  9. Thomasaroub 2023 年 1 月 29 日 08:38 回复
    Microsoft Edge 90.0.818.46 Microsoft Edge 90.0.818.46 Windows 10 Windows 10
    Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36 Edg/90.0.818.46

    https://doxycycline.science/# doxycycline hydrochloride 100mg

  10. RonaldObevy 2023 年 1 月 29 日 09:31 回复
    Google Chrome 90.0.4430.72 Google Chrome 90.0.4430.72 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36

    amoxicillin buy no prescription order amoxil
    https://doxycycline.science/# buy doxycycline hyclate 100mg without a rx
    amoxil buy amoxicillin online mexico

  11. Ralphartew 2023 年 1 月 29 日 12:09 回复
    Microsoft Edge 90.0.818.46 Microsoft Edge 90.0.818.46 Windows 10 Windows 10
    Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36 Edg/90.0.818.46

    minocycline warnings ivermectin 9 mg or ivermectin lotion for lice
    http://www.brianlowe.com/__media__/js/netsoltrademark.php?d=stromectol.science ivermectin topical
    ivermectin 6 stromectol ivermectin tablets and ivermectin 4 tablets price buy minocycline 50mg otc

  12. AlbertLal 2023 年 1 月 29 日 16:31 回复
    Google Chrome 90.0.4430.86 Google Chrome 90.0.4430.86 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.86 Safari/537.36

    zithromax capsules 250mg zithromax cost australia or zithromax antibiotic
    http://izv.yasha.com/__media__/js/netsoltrademark.php?d=zithromax.science zithromax for sale 500 mg
    zithromax 500 mg lowest price drugstore online zithromax 500 without prescription and zithromax prescription online zithromax 500mg

  13. Rafaelkek 2023 年 1 月 30 日 04:59 回复
    Google Chrome 90.0.4430.72 Google Chrome 90.0.4430.72 Windows 7 Windows 7
    Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36

    https://zithromax.science/# zithromax capsules australia
    generic amoxil online amoxicillin discount coupon

  14. RonaldWorce 2023 年 1 月 30 日 05:15 回复
    Microsoft Edge 90.0.818.42 Microsoft Edge 90.0.818.42 Windows 8.1 x64 Edition Windows 8.1 x64 Edition
    Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36 Edg/90.0.818.42

    buy doxycycline cheap doxy 200
    https://amoxil.science/# purchase amoxicillin online without prescription
    buy zithromax zithromax without prescription

  15. RonaldObevy 2023 年 1 月 30 日 08:55 回复
    Google Chrome 90.0.4430.85 Google Chrome 90.0.4430.85 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36

    zithromax online paypal buy zithromax
    https://stromectol.science/# minocycline coupon
    cheap zithromax zithromax 600 mg tablets

  16. DavidHap 2023 年 1 月 30 日 19:45 回复
    Google Chrome 90.0.4430.85 Google Chrome 90.0.4430.85 Windows 7 x64 Edition Windows 7 x64 Edition
    Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36

    order amoxicillin online amoxicillin 825 mg or amoxicillin 500 mg tablet price
    http://leoneindustries.us/__media__/js/netsoltrademark.php?d=amoxil.science amoxicillin 875 125 mg tab
    amoxicillin price without insurance where can i buy amoxocillin and amoxicillin 500mg capsules antibiotic amoxicillin price without insurance

  17. Thomasaroub 2023 年 1 月 31 日 00:38 回复
    Google Chrome 90.0.4430.85 Google Chrome 90.0.4430.85 Windows 7 x64 Edition Windows 7 x64 Edition
    Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36

    https://doxycycline.science/# doxycycline 150 mg

  18. Jesselemia 2023 年 1 月 31 日 03:09 回复
    Microsoft Edge 90.0.818.42 Microsoft Edge 90.0.818.42 Windows 8.1 x64 Edition Windows 8.1 x64 Edition
    Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36 Edg/90.0.818.42

    doxycycline hydrochloride 100mg online doxycycline or buy cheap doxycycline
    http://unofficialtrailguide.info/__media__/js/netsoltrademark.php?d=doxycycline.science buy doxycycline cheap
    doxycycline hyc doxycycline 150 mg and doxycycline 500mg doxycycline hydrochloride 100mg

  19. Rafaelkek 2023 年 1 月 31 日 08:48 回复
    Google Chrome 90.0.4400.8 Google Chrome 90.0.4400.8 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4400.8 Safari/537.36

    https://doxycycline.science/# 200 mg doxycycline
    buy amoxil amoxicillin script

  20. RonaldWorce 2023 年 1 月 31 日 11:00 回复
    Google Chrome 90.0.4430.72 Google Chrome 90.0.4430.72 Windows 8.1 Windows 8.1
    Mozilla/5.0 (Windows NT 6.3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36

    doxycycline 100mg tablets how to buy doxycycline online
    https://stromectol.science/# stromectol cvs
    cheap doxycycline online where can i get doxycycline

  21. AlbertLal 2023 年 1 月 31 日 16:13 回复
    Google Chrome 90.0.4430.85 Google Chrome 90.0.4430.85 Windows 7 x64 Edition Windows 7 x64 Edition
    Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36

    zithromax 1000 mg pills buy zithromax 1000mg online or buy zithromax 1000mg online
    http://mvbtrust.com/__media__/js/netsoltrademark.php?d=zithromax.science where can you buy zithromax
    order zithromax without prescription zithromax drug and zithromax 1000 mg online zithromax capsules

  22. Hampannef 2023 年 1 月 31 日 16:23 回复
    Google Chrome 90.0.4430.72 Google Chrome 90.0.4430.72 Windows 7 Windows 7
    Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36

    tamoxifen and covid vaccine com 20 E2 AD 90 20Beta 20Blocker 20Bisoprolol 20Viagra 20 20Buy 20Generic 20Viagra 20Online 20Canada buy generic viagra online canada Justice Antonin Scalia, one of the court s most conservative, sided with the majority in the Arizona v

  23. RonaldObevy 2023 年 1 月 31 日 18:01 回复
    Google Chrome 90.0.4430.85 Google Chrome 90.0.4430.85 Windows 7 x64 Edition Windows 7 x64 Edition
    Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36

    can you buy zithromax online buy zithromax online
    https://amoxil.science/# amoxicillin 30 capsules price
    amoxicillin without prescription amoxicillin 500 mg without prescription

  24. GustavoSwemi 2023 年 1 月 31 日 19:16 回复
    Google Chrome 90.0.4430.93 Google Chrome 90.0.4430.93 Windows 7 Windows 7
    Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36

    order doxycycline 100mg without prescription doxycycline prices or doxycycline 50mg
    http://hawaiikaideep.com/__media__/js/netsoltrademark.php?d=doxycycline.science doxycycline 200 mg
    doxycycline mono 100mg doxycycline and doxycycline hydrochloride 100mg doxy

  25. Ralphartew 2023 年 1 月 31 日 19:21 回复
    Google Chrome 90.0.4430.85 Google Chrome 90.0.4430.85 Windows 7 x64 Edition Windows 7 x64 Edition
    Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36

    buy ivermectin cream buy minocycline 100mg tablets or stromectol tab
    http://primoshoagies.com/__media__/js/netsoltrademark.php?d=stromectol.science purchase ivermectin
    minocycline 50mg without doctor is minocycline an antibiotic and does minocycline work for acne ivermectin 50 mg

  26. roocratte 2023 年 1 月 31 日 21:36 回复
    Microsoft Edge 90.0.818.49 Microsoft Edge 90.0.818.49 Windows 8.1 x64 Edition Windows 8.1 x64 Edition
    Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36 Edg/90.0.818.49

    Следват 5 вълнуващи седмици в казино Sesame. Всяка седмица в периода от 29.08.2022 до 02.10.2022 г. ви очакват не един, а цели два турнира – един за Казино и един за Спорт. И може да участвате и в двата едновременно. Всяка седмица екипа на казиното ще раздава 650 награди под формата на бонус средства с общ награден фонд от 40 000 лв. в казино и спорт турнирите. Разбира се платформата за залози има и сезам мобилна версия съвместима с всички мобилни устройства и таблети. За сега Sesame все още няма официално пуснато мобилно приложение но ако от бранда желаят да спазват новите тенденции и да бъдат конкуренти ще бъде необходимо да пуснат и апликация за андройд и айфоун. https://forums.it-alfa.com/profile/carriallen56021/ „Топ 10 на седмицата“ в онлайн казиното на efbet Ако все още не сте добре запознати с ротативките, ние ще ви предложим няколко страхотни слота на топ производители, които ще срещнете в много от онлайн казината. Съветваме всеки един начинаещ играч да започне от тях, тъй като това са доказали са се ротативки, които много често носят огромни печалби на клиентите на онлайн казината. Това са игри на късмета, така че няма изпитана стратегия, която да Ви донесе печалба. Все пак трябва да се придържате към икономичните залози в началото на игрите, и да преминавате към по-високи залагания, ако печелите.

  27. Rafaelkek 2023 年 2 月 1 日 08:49 回复
    Microsoft Edge 90.0.818.49 Microsoft Edge 90.0.818.49 Windows 7 x64 Edition Windows 7 x64 Edition
    Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36 Edg/90.0.818.49

    https://doxycycline.science/# doxycycline hyclate 100 mg cap
    doxycycline online doxycycline order online

  28. JamesLam 2023 年 2 月 2 日 16:59 回复
    Google Chrome 90.0.4430.85 Google Chrome 90.0.4430.85 Windows 7 Windows 7
    Mozilla/5.0 (Windows NT 6.1; ) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36

    Top 100 Searched Drugs. Everything about medicine.
    cost of ivermectin 1% cream
    Everything information about medication. Medscape Drugs & Diseases.

  29. RonaldWorce 2023 年 2 月 3 日 03:51 回复
    Microsoft Edge 90.0.818.42 Microsoft Edge 90.0.818.42 Windows 8.1 x64 Edition Windows 8.1 x64 Edition
    Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36 Edg/90.0.818.42

    Read information now. Drug information.
    ivermectin 3 mg tablet dosage
    Drugs information sheet. Everything what you want to know about pills.

  30. Kevinkex 2023 年 2 月 3 日 04:20 回复
    Google Chrome 90.0.4430.93 Google Chrome 90.0.4430.93 Windows 10 Windows 10
    Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36

    Some are medicines that help people when doctors prescribe. Medicament prescribing information.
    https://stromectolst.com/# stromectol oral
    Top 100 Searched Drugs. Read information now.

  31. Zacharyanold 2023 年 2 月 3 日 08:02 回复
    Google Chrome 90.0.4430.86 Google Chrome 90.0.4430.86 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.86 Safari/537.36

    Comprehensive side effect and adverse reaction information. earch our drug database.
    ivermectin where to buy
    Everything what you want to know about pills. Everything about medicine.

  32. JamesLam 2023 年 2 月 3 日 10:35 回复
    Google Chrome 90.0.4430.72 Google Chrome 90.0.4430.72 Windows 7 x64 Edition Windows 7 x64 Edition
    Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36

    Drug information. Some are medicines that help people when doctors prescribe.
    ivermectin lotion
    Everything information about medication. Get here.

  33. Williamweize 2023 年 2 月 3 日 12:19 回复
    Microsoft Edge 90.0.818.46 Microsoft Edge 90.0.818.46 Windows 8.1 x64 Edition Windows 8.1 x64 Edition
    Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36 Edg/90.0.818.46

    ivermectin eye drops stromectol ivermectin tablets or minocycline for rheumatoid arthritis
    http://zeropeg.com/__media__/js/netsoltrademark.php?d=stromectol.science stromectol where to buy
    ivermectin 1% cream generic minocycline indications and ivermectin lotion for lice minocycline warnings

  34. RonaldWorce 2023 年 2 月 4 日 04:13 回复
    Google Chrome 90.0.4430.85 Google Chrome 90.0.4430.85 Windows 7 x64 Edition Windows 7 x64 Edition
    Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36

    Best and news about drug. Commonly Used Drugs Charts.
    https://stromectolst.com/# ivermectin 50ml
    drug information and news for professionals and consumers. Drugs information sheet.

  35. Kevinkex 2023 年 2 月 4 日 04:51 回复
    Google Chrome 90.0.4430.93 Google Chrome 90.0.4430.93 Windows 8.1 x64 Edition Windows 8.1 x64 Edition
    Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36

    Read information now. Medscape Drugs & Diseases.
    stromectol coronavirus
    Commonly Used Drugs Charts. Learn about the side effects, dosages, and interactions.

  36. Zacharyanold 2023 年 2 月 4 日 13:51 回复
    Google Chrome 90.0.4430.72 Google Chrome 90.0.4430.72 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36

    safe and effective drugs are available. Comprehensive side effect and adverse reaction information.
    stromectol
    What side effects can this medication cause? Comprehensive side effect and adverse reaction information.

  37. Williamweize 2023 年 2 月 4 日 14:19 回复
    Google Chrome 90.0.4430.93 Google Chrome 90.0.4430.93 Windows 7 Windows 7
    Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36

    stromectol south africa ivermectin iv or minocycline 50 mg pills online
    http://virtualinfrastructureutility.biz/__media__/js/netsoltrademark.php?d=stromectol.science stromectol 6 mg dosage
    minocycline drug minocycline uses and minocycline 50mg over the counter ivermectin drug

  38. AnthonyAcacy 2023 年 2 月 4 日 16:33 回复
    Microsoft Edge 90.0.818.46 Microsoft Edge 90.0.818.46 Windows 8.1 x64 Edition Windows 8.1 x64 Edition
    Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36 Edg/90.0.818.46

    ivermectin cost uk stromectol order or buy ivermectin nz
    http://benkliper.com/__media__/js/netsoltrademark.php?d=stromectolst.com ivermectin 9 mg
    stromectol order cost of stromectol medication and ivermectin pill cost buy stromectol online

  39. Bennieges 2023 年 2 月 4 日 16:36 回复
    Google Chrome 90.0.4430.93 Google Chrome 90.0.4430.93 Windows 10 Windows 10
    Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36

    stromectol online stromectol australia or stromectol 12mg
    http://initsimage.net/__media__/js/netsoltrademark.php?d=stromectolst.com&cache=yes stromectol 3mg
    ivermectin 6mg stromectol coronavirus and ivermectin cream uk ivermectin 3 mg tablet dosage

  40. Kevinkex 2023 年 2 月 4 日 23:52 回复
    Google Chrome 90.0.4430.72 Google Chrome 90.0.4430.72 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36

    Everything about medicine. Comprehensive side effect and adverse reaction information.
    cost of stromectol medication
    Commonly Used Drugs Charts. Some trends of drugs.

  41. Jesselemia 2023 年 2 月 5 日 02:40 回复
    Google Chrome 90.0.4430.85 Google Chrome 90.0.4430.85 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36

    Commonly Used Drugs Charts. Comprehensive side effect and adverse reaction information.
    https://stromectolst.com/# stromectol uk buy
    Top 100 Searched Drugs. earch our drug database.

  42. RonaldWorce 2023 年 2 月 5 日 03:52 回复
    Google Chrome 90.0.4400.8 Google Chrome 90.0.4400.8 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4400.8 Safari/537.36

    Get here. Commonly Used Drugs Charts.
    https://stromectolst.com/# stromectol price
    Get information now. Top 100 Searched Drugs.

  43. Kevinkex 2023 年 2 月 5 日 15:11 回复
    Microsoft Edge 90.0.818.39 Microsoft Edge 90.0.818.39 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36 Edg/90.0.818.39

    Medscape Drugs & Diseases. Drugs information sheet.
    https://stromectolst.com/# ivermectin australia
    Learn about the side effects, dosages, and interactions. drug information and news for professionals and consumers.

  44. Williamweize 2023 年 2 月 5 日 16:36 回复
    Google Chrome 90.0.4430.93 Google Chrome 90.0.4430.93 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36

    buy minocycline 50 mg otc minocycline pill or minocycline 100 mg tabs
    http://sanitarystandardcorp.com/__media__/js/netsoltrademark.php?d=stromectol.science minocycline 50mg over the counter
    ivermectin 50mg/ml ivermectin brand name and ivermectin 0.5% lotion price of stromectol

  45. Zacharyanold 2023 年 2 月 5 日 23:41 回复
    Google Chrome 90.0.4427.0 Google Chrome 90.0.4427.0 Windows 7 x64 Edition Windows 7 x64 Edition
    Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4427.0 Safari/537.36

    п»їMedicament prescribing information. Read now.
    https://stromectolst.com/# ivermectin uk coronavirus
    Top 100 Searched Drugs. Some are medicines that help people when doctors prescribe.

  46. RonaldWorce 2023 年 2 月 6 日 04:24 回复
    Google Chrome 90.0.4430.72 Google Chrome 90.0.4430.72 Windows 7 x64 Edition Windows 7 x64 Edition
    Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36

    Best and news about drug. Medscape Drugs & Diseases.
    https://stromectolst.com/# stromectol tablets uk
    Best and news about drug. safe and effective drugs are available.

  47. Kevinkex 2023 年 2 月 6 日 06:36 回复
    Google Chrome 90.0.4430.85 Google Chrome 90.0.4430.85 Windows 7 x64 Edition Windows 7 x64 Edition
    Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36

    Drug information. Get information now.
    buy ivermectin canada
    Medscape Drugs & Diseases. drug information and news for professionals and consumers.

  48. Stevieomica 2023 年 2 月 6 日 12:00 回复
    Google Chrome 90.0.4430.72 Google Chrome 90.0.4430.72 Windows 8.1 x64 Edition Windows 8.1 x64 Edition
    Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36

    ivermectin 6 stromectol 6 mg tablet or ivermectin purchase
    http://sage-advertising.com/__media__/js/netsoltrademark.php?d=stromectolst.com stromectol how much it cost
    stromectol ivermectin buy stromectol cvs and ivermectin 5 stromectol 3mg

  49. Williamweize 2023 年 2 月 6 日 19:10 回复
    Google Chrome 90.0.4430.85 Google Chrome 90.0.4430.85 Windows 7 Windows 7
    Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36

    stromectol lotion minocycline tablets or what is minocycline 50 mg used for
    http://essentialtruths.net/__media__/js/netsoltrademark.php?d=stromectol.science buy minocycline 50 mg otc
    generic ivermectin ivermectin uk coronavirus and ivermectin brand name ivermectin 3mg pill

  50. Jesselemia 2023 年 2 月 6 日 22:35 回复
    Google Chrome 90.0.4430.72 Google Chrome 90.0.4430.72 Windows 7 x64 Edition Windows 7 x64 Edition
    Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36

    Get warning information here. Actual trends of drug.
    stromectol buy
    Read now. Definitive journal of drugs and therapeutics.

  51. Kevinkex 2023 年 2 月 7 日 03:20 回复
    Google Chrome 90.0.4400.8 Google Chrome 90.0.4400.8 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4400.8 Safari/537.36

    Best and news about drug. Commonly Used Drugs Charts.
    ivermectin over the counter
    Drugs information sheet. All trends of medicament.

  52. AnthonyAcacy 2023 年 2 月 7 日 06:48 回复
    Google Chrome 90.0.4430.93 Google Chrome 90.0.4430.93 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36

    stromectol tablet 3 mg ivermectin 6 mg tablets or stromectol prices
    http://globalsquare.biz/__media__/js/netsoltrademark.php?d=stromectolst.com ivermectin 2ml
    ivermectin pill cost ivermectin 50ml and stromectol for sale stromectol cream

  53. RonaldWorce 2023 年 2 月 7 日 09:02 回复
    Google Chrome 90.0.4430.85 Google Chrome 90.0.4430.85 Windows 10 x64 Edition Windows 10 x64 Edition
    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36

    Read here. safe and effective drugs are available.
    generic ivermectin for humans
    Get information now. Everything what you want to know about pills.

  54. WillieElamn 2023 年 2 月 7 日 16:45 回复
    Google Chrome 90.0.4430.85 Google Chrome 90.0.4430.85 Windows 8.1 x64 Edition Windows 8.1 x64 Edition
    Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36

    Long-Term Effects. Get information now.
    where to get cheap mobic price
    Get information now. Drugs information sheet.

发表回复

您的电子邮箱地址不会被公开。 必填项已用*标注

 
Ɣ回顶部