{"id":12463,"date":"2021-08-19T17:44:47","date_gmt":"2021-08-19T17:44:47","guid":{"rendered":"http:\/\/ismiletechnologies.com\/?p=12463"},"modified":"2022-10-06T17:12:24","modified_gmt":"2022-10-06T11:42:24","slug":"troubleshooting-in-apache-cassandra","status":"publish","type":"post","link":"https:\/\/ismiletechnologies.com\/en_us\/cloud-services\/troubleshooting-in-apache-cassandra\/","title":{"rendered":"Troubleshooting in Apache Cassandra"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Apache Cassandra is a distributed system, limiting the number of requests per node and balancing load better. But in any distributed system, when you are trying to search the root cause of any problem, you need to narrow down where the problem is occurring&nbsp;and affecting the entire cluster. In Apache Cassandra, it denotes identifying the nodes or instances that are responsible for the problem in the entire structure. Only after that can you fix the problem and have a resolution.&nbsp;&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">An effective strategy for identifying where the problem is and for identifying the scope of the problem, you need to leverage metrics data to gain focused&nbsp;insights and log analysis. This helps in identifying the root cause quickly. Cassandra provides users with different metrics that enable incident response. By analyzing these metrics, the existence of the problem and their accurate location (either a node or data center) can be identified.&nbsp;&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s discuss the feasible metrics that can be used for root cause analysis.&nbsp;&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Metrics related to client requests- The metrics related to client requests provide insights into failures, timeouts, request type, latency and throughput, etc.&nbsp;&nbsp;<\/li><li>Metrics related to the table- The metrics which allow for accessing the performance of the tables include&nbsp;&nbsp;<\/li><li>MemtableOnHeapSize- Amount of data residing on the heap in the&nbsp;memtable&nbsp;&nbsp;<\/li><li>MemtableOffHeapSize-Amount of data residing off-heap in the&nbsp;memtable&nbsp;&nbsp;<\/li><li>MemtableLiveDataSize- mount of stored live data in the&nbsp;memtable&nbsp;&nbsp;<\/li><li>AllMemtablesOnHeapSize- Amount of data stored in&nbsp;memtables&nbsp;&nbsp;<\/li><li>AllMemtablesLiveDataSize- Amount of live data in&nbsp;memtables&nbsp;including pending flush and 2i&nbsp;memtables&nbsp;&nbsp;<\/li><li>MemtableColumnsCount- The number of columns in&nbsp;memtable&nbsp;&nbsp;<\/li><li>MemtableSwitchCount- Number of times&nbsp;memtable&nbsp;has been switched out because of flush&nbsp;&nbsp;<\/li><li>CompressionRatio- The compression ratio for all&nbsp;SSTables&nbsp;&nbsp;<\/li><li>ReadLatency- It denotes the local read latency for the table.&nbsp;&nbsp;<\/li><li>RangeLatency- It denotes the local range latency for the table.&nbsp;&nbsp;<\/li><li>PendingFlushes-It denotes the average number of flush tasks pending for&nbsp;the table.&nbsp;&nbsp;<\/li><li>BytesFlushed- It denotes the number of flushed bytes since server restart&nbsp;&nbsp;<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Many other metrics are considered for knowing the performance of the tables in the database.&nbsp;&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Similarly, there are many other\u00a0keyspace\u00a0metrics to access the performance of the\u00a0keyspaces.\u00a0\u00a0<\/p>\n\n\n\t\t<div data-elementor-type=\"section\" data-elementor-id=\"33253\" class=\"elementor elementor-33253\" data-elementor-post-type=\"elementor_library\">\n\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-1f27ef0 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"1f27ef0\" data-element_type=\"section\" data-e-type=\"section\" data-settings=\"{&quot;background_background&quot;:&quot;gradient&quot;}\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-c65c317\" data-id=\"c65c317\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-inner-section elementor-element elementor-element-dc3c880 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"dc3c880\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-33 elementor-inner-column elementor-element elementor-element-3248be5\" data-id=\"3248be5\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-5886bd6 elementor-widget elementor-widget-heading\" data-id=\"5886bd6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Ready to experience the full power of cloud technology?<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1f56c4e elementor-widget elementor-widget-text-editor\" data-id=\"1f56c4e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"TextRun Highlight SCXW216552313 BCX0\" lang=\"EN-IN\" xml:lang=\"EN-IN\" data-contrast=\"none\"><span class=\"NormalTextRun SCXW216552313 BCX0\">Our cloud experts will speed up cloud deployment, and make your business more efficient.\u00a0<\/span><\/span><span class=\"EOP SCXW216552313 BCX0\" data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:160,&quot;335559740&quot;:259}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-33 elementor-inner-column elementor-element elementor-element-4b7dd61\" data-id=\"4b7dd61\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-81f081a elementor-align-center elementor-widget elementor-widget-lottie\" data-id=\"81f081a\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;source&quot;:&quot;external_url&quot;,&quot;source_external_url&quot;:{&quot;url&quot;:&quot;https:\\\/\\\/assets4.lottiefiles.com\\\/packages\\\/lf20_kkesf8mx.json&quot;,&quot;is_external&quot;:&quot;&quot;,&quot;nofollow&quot;:&quot;&quot;,&quot;custom_attributes&quot;:&quot;&quot;},&quot;loop&quot;:&quot;yes&quot;,&quot;play_speed&quot;:{&quot;unit&quot;:&quot;px&quot;,&quot;size&quot;:0.9,&quot;sizes&quot;:[]},&quot;lazyload&quot;:&quot;yes&quot;,&quot;link_to&quot;:&quot;none&quot;,&quot;trigger&quot;:&quot;arriving_to_viewport&quot;,&quot;viewport&quot;:{&quot;unit&quot;:&quot;%&quot;,&quot;size&quot;:&quot;&quot;,&quot;sizes&quot;:{&quot;start&quot;:0,&quot;end&quot;:100}},&quot;start_point&quot;:{&quot;unit&quot;:&quot;%&quot;,&quot;size&quot;:0,&quot;sizes&quot;:[]},&quot;end_point&quot;:{&quot;unit&quot;:&quot;%&quot;,&quot;size&quot;:100,&quot;sizes&quot;:[]},&quot;renderer&quot;:&quot;svg&quot;}\" data-widget_type=\"lottie.default\">\n\t\t\t\t\t<div class=\"e-lottie__container\"><div class=\"e-lottie__animation\"><\/div><\/div>\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-33 elementor-inner-column elementor-element elementor-element-ac6262c\" data-id=\"ac6262c\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-e19f719 elementor-align-right elementor-mobile-align-left elementor-tablet-align-right elementor-widget elementor-widget-button\" data-id=\"e19f719\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t\t\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-xs\" href=\"#elementor-action%3Aaction%3Dpopup%3Aopen%26settings%3DeyJpZCI6IjI5OTEiLCJ0b2dnbGUiOnRydWV9\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Start Today !<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t\n\n\n\n<p class=\"wp-block-paragraph\">Collecting Cassandra metrics is generally done with&nbsp;&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><strong><em>Node tool<\/em><\/strong>&nbsp;<\/li><li><strong><em>JConsole&nbsp;<\/em><\/strong>&nbsp;<\/li><li><strong><em>Integrations of JMX\/Metrics<\/em><\/strong>&nbsp;<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Node tool<\/strong>\u2013 This is a command-line tool. The node tool runs straight from an operational node. It helps to view detailed metrics for tables along with server metrics and compaction statistics.&nbsp;&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">There are many steps for troubleshooting in Cassandra.&nbsp;&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Identify the faulty nodes&nbsp;&nbsp;<\/li><li>The nodes consist of multiple&nbsp;keyspaces.&nbsp;One needs to find the faulty&nbsp;keyspace&nbsp;within the node.&nbsp;&nbsp;<\/li><li>The next step is finding out errors or defects in the source code. One can dry run the whole code or find out which part is faulty.&nbsp;&nbsp;<\/li><li>There may be changes in properties or versions leading to errors. For mitigating this, the user needs to script the source code according to the current properties.&nbsp;&nbsp;<\/li><li>The execution files of Cassandra, CQL, and the APIs may also malfunction.&nbsp;&nbsp;<\/li><li>After finding the errors and the sources of errors, the next step is to fix them by debugging or fixing the errors or writing the entire code.&nbsp;&nbsp;<\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Apache Cassandra is a distributed system, limiting the number of requests per node and balancing load better. But in any distributed system, when you are trying to search the root cause of any problem, you need to narrow down where the problem is occurring&nbsp;and affecting the entire cluster. In Apache Cassandra, it denotes identifying the [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":13139,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[81],"tags":[180,181],"class_list":["post-12463","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cloud-services","tag-apache","tag-cassandra"],"_links":{"self":[{"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/posts\/12463","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/comments?post=12463"}],"version-history":[{"count":3,"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/posts\/12463\/revisions"}],"predecessor-version":[{"id":33658,"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/posts\/12463\/revisions\/33658"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/media\/13139"}],"wp:attachment":[{"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/media?parent=12463"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/categories?post=12463"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ismiletechnologies.com\/en_us\/wp-json\/wp\/v2\/tags?post=12463"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}